Getting started with Azure Arc-enabled servers

Azure Arc, have you tried it out already?

The advertisement of Azure Arc is great:

Azure Arc enables you to extend Azure management capabilities to hybrid environments, including on-premises datacenters and third-party cloud providers. You can use Azure Arc to manage and configure your Windows and Linux server machines and Kubernetes clusters that are hosted outside of Azure. You can also use Azure Arc to introduce Azure data services to hybrid environments.

So, we can manage server machines, Kubernetes, and data services, all over the world as if these live in Azure:

Remote server management, this is a very interesting feature from an IoT point of view because an IoT Developer wants to have control over all edge devices without having to manage them only by visiting them.

Zero-touch maintenance, could that be done using Azure Arc?

Today, we will look at a small but very important part of Azure Arc, Azure Arc-enabled servers, so we can answer that question.

Let’s see how easy it is.

Azure Arc-enabled services are something that could go very well with Azure IoT Edge:

Azure Arc-enabled servers lets you manage Windows and Linux physical servers and virtual machines hosted outside of Azure, on your corporate network, or other cloud provider.

We run either physical Linux or Windows (EFlow) edge devices but the Azure IoT hub only lets us manage the containers, not the Docker runtime or the operating system.

The OSConfig tool is also nice but does not give us a separate maintenance channel for the device, in case Azure IoT Edge fails.

Looking at the pricing, the basic features are not that expensive too:

Free inventory and management? Does it actually support SSH?

And it even comes with update management?

This is too good to be true!

Let’s check out if it works as well as advertised.

Prerequisites

For this demonstration, I use a moderate IPC from Advantech, the UNO 1273GJ, running on a Celeron processor with 4GB of memory:

I used a wired connection and installed Ubuntu 22.04.4 LTS, the server version on it.

This is a nice device for regular Azure IoT Edge purposes, simple but not too light and still has a snappy performance.

Note: I also tried an UNO 2271G having a very old Atom processor and only 4GB of RAM. Azure Arc works as a charm there too.

Notice I did not install an SSH server on it during the default installation.

You can play along with a similar device if you want.

Adding a machine registration

We want to arc-enable an IPC using Azure Arc so within the portal, check out Azure Arc.

In the Machines section, you can add a machine:

When you follow that option, you are asked to generate a script so an agent can be installed on our device (a single server):

Note: this script will ask for your Azure login during the installation process. Check the other options for other ways to authenticate (like a service principle).

This agent will then create an (outbound) connection with the server machine registration in the Azure portal.

To create the script, we first need to add a few details:

We create a script for a Linux device and we will make use of the (already quite secure) public endpoint of Azure Arc.

Notice the resource group to manage multiple related machines if needed because each device you add will end up in some resource group so you can order them that way.

Select the region closest to your device so you limit the network lag.

A more feasible way to manage devices at scale is by adding tags:

These extra tags are used to filter and search for subsets of devices living somewhere in this world.

Once the tags are added, the script will appear:

The registration is done but we are not ready yet: the script must be executed.

Install the agent

Make a copy of the script provided and place it on the IPC device to be executed locally.

Because I did not install SSH (so I can not copy the script to the UNO IPC and execute it on the prompt), I had to copy the file from a USB stick before it could be executed:

Using these commands, I mounted and dismounted the USB stick first to copy the script file.

Notice you need Super User rights to execute the file so it basically installs the agent with the correct parameters.

During the installation, you will be asked to log in to Azure and supply your (admin) credentials to prove you have access to your Azure subscription using the right role.

The address is https://microsoft.com/devicelogin. There you fill in a code generated on the device like ‘CH9H4JUCA’.

This dialog asks you to log in with your Azure credentials and asks for confirmation:

Once you supply your credentials successfully, the agent is installed and connected:

Check the Azure portal resource group. You will find your machine (as a ‘hybrid resource’) there:

Notice it is connected and it shows some device registration information taken from the IPC:

Also, check out all the tags for this machine. We filled them in during the creation of the script. These can also be used for billing (if you add some kind of cost center tag to your machines).

What has happened?

The agent has opened an encrypted communication tunnel over an outbound TCP connection over port 443 towards the Azure control plane and the agent keeps this connection open.

From here, several options are available:

• You can see the compliance of your machine configuration
• You can connect using SSH
• Update settings
• etc.

We will check a few of them and see what is offered.

Notice the agent daemon application will be updated if a ‘sudo apt update / sudo apt upgrade’ is executed on the IPC.

Machine configuration information

The first feature we visit is the machine configuration.

This is a list of configurations assigned to your server. Configurations can be assigned in three ways: through Azure Policy, manually, or system-assigned. Manual assignments represent custom configurations assigned to this server. Through manual configurations you can edit the version, configuration type and custom URI.

This list of settings shows what is not compliant on our local machine, compared to the Azure policies:

In this case, this is the set related to the Azure Linux baseline.

Check out the documentation for more details.

SSH connection

Next, we check out the capability for creating an SSH connection from the Azure cloud to our device, wherever it is located in the world.

It is advertised that ‘No public IP address or open SSH ports’ are required. Still, because this connection relies on an SSH server running on the machine, I do not recommend this connection for those devices connected to the internet without a proper firewall in between or with explicit SSH inbound rules. Please test your connection if you want to experiment yourself.

Although this also works for Virtual Machines, it’s not the best solution for VMs running in Azure. Please check out Azure Bastion for those Azure Virtual Machines.

So set up an SSH connection, start visiting with the Connect page:

You can either connect using an Entra ID, an SSH key, or a name/password combination.

Just to demonstrate this service, I chose the name/password combination. Notice I have to fill in the user name. This option is only available for Linux devices.

This provides me a PowerShell command and parameters I have to execute within the Azure CLI when being logged in to the same Azure subscription as the machine is living in:

The Azure CLI is something you can download and install on your machine (in PowerShell). You can also make use of the Windows Terminal application with built-in support for the Azure CLI or you can open the Azure Portal shell.

The first time I ran this command for my UNO IPC, I got a message that SSH was not supported on my Linux machine yet. That was the correct observation, I did not enable that feature while executing the Linux installation.

To make use of the SHH connection, the SSH server was installed in the end by the script:

Finally, I was asked to provide the password and I was able to log in to the prompt of my IPC:

This is a very convenient way to connect securely at the level of the operating system on your Edge device.

This connection is available right after a startup or reboot (give it a minute or so :-)). A local login is not needed.

Fun fact, this SSH communication is using the Azure Arc outbound TCP tunnel created by the agent. It is based on the Azure Relay service, a hidden gem in Azure because it can create secure tunnels all over the internet and beyond. This is also the magic used to support the communication for all other extensions (like Azure IoT Operations).

Check out the documentation for more details about the SSH connection. There is also a troubleshooting guide.

Update settings

Finally, we look into the Update management feature.

You can use Update Management in Azure Automation to manage operating system updates for your Windows and Linux virtual machines in Azure, physical or VMs in on-premises environments, and in other cloud environments. You can quickly assess the status of available updates and manage the process of installing required updates for your machines reporting to Update Management.

All you need to do first is to enable this feature:

Once this is done, once in a while your machine is audited for possible updates.

So you get a message like this when the device has no updates:

You can trigger an assessment by hand too, in the portal (this will take some time).

This will result in an overview like this although there are no patches advertised yet:

Note: During the writing of this blog post, I did not encounter any updates.

When there are updates, you see this:

You can even install a one-time update:

Check the documentation for details, including related automation features.

Filter

Do you remember I added a few tags so I could filter the right machine?

If you have added all your machines in different resource groups, all these machines can still be listed on one page:

As you can see, it comes with a query button.

This brings us to a very elaborate LogAnalytics query written in KQL.

If you want to filter on the tags, check out the line below where I filter for machines in the data center called ‘home’:

The result is a subset of all machines honoring this tag filter.

Conclusion

We have learned how to set up a secure connection between an Advantech Uno Industrial PC running Linux and the Azure cloud.

I say this offers zero-touch management. Azure Arc is a great way to manage your local (IoT edge) devices wherever they are in the world. This is true, even if this means other clouds.

Using the extensions, new ways of (automated) management can be added, including managing Kubernetes clusters, for example.

Azure Arc is also the foundation for Azure IoT Operations aka Azure IoT Edge 2.0.

Here, I demonstrated Azure Arc on the latest LTS Ubuntu version. Check this list of supported operating systems. Keep in mind this is great for servers. If you want to manage clients, check out Intune.

Note: Yocto Linux is not supported. If you are interested, share it as an idea at Share your ideas.

I also demonstrated this with a public Azure endpoint for Azure Arc. If needed, you could make use of a private link with Azure Arc if this is required for even more security.

Keep in mind this Azure Arc works best on an ‘almost always connected device’. Air-gapped devices are out of scope. If you use some router in between, try to find out who is responsible for turning on the connection to the internet.

What we have seen above, part of the core control plane of Azure Arc, is mostly free. Check out the pricing page for all details. The rule of thumb is that if you pay for the service on Azure, you probably also pay for it in the context of Azure Arc (eg. Defender, log Analytics, etc.).

For a more elaborate example, check out this Arc Jumpstart using an Azure service principal for authentication.

Or pick one of the other examples supporting eg. Amazon, Google, VMWare, or HashiCorp.