UWP app as Virtual Shields for Arduino

Are you in the position of not having a screen, GPS module, internet connection, GSM/SMS module, Gyro module, compass module, microphone module, etc. for your Arduino?

Normally you went out surfing to your favorite internet dealer for the components to buy them. And these are all pretty expensive modules. If you choose to buy them as a shield, you will have to pay even more.

But if you own a Lumia smart phone, then you could be lucky. Check this site if your Lumia is compatible with Windows 10 Mobile. Why? Because these devices support UWP apps and Microsoft has provided this Virtual Shields app.

“The Virtual Shields for Arduino application enables developers and makers to use a Windows 10 device as a sensor shield for their Arduino board. An Arduino sketch can now make use of any of the sensor and actuator assets of a Windows device…”

So an Arduino can control and use the sensors of your Lumia, that’s interesting!

If you want to look at the code of this UWP, get it from here. But if you get the app from the Windows 10 app store you are ready on the Lumia (it’s now a Dumb phone?). This will turn your Lumia into a slave for the Arduino master.

Over to the Arduino. You have do two things first, In the Arduino IDE include:

  1. the ArduinoJson library
  2. the Windows Virtual Shields for Arduino library

Then connect a Bluetooth module to your Arduino to port 0 and 1 (Make sure your  Lumia is paired to this Bluetooth module):

vandeVelde 01 BasicFirmata

Now you are ready for your first Arduino sketch using the screen shield:

#include <ArduinoJson.h>
#include <VirtualShield.h>
#include <Text.h>

VirtualShield shield;
Text screen = Text(shield);

void setup()
  screen.printAt(0, "Hello");
  screen.printAt(2, "my Virtual Shields");
  screen.printAt(3, "for Arduino");

void loop()

Put this sketch on your Arduino and start up the UWP app. Because of the usage of a cheap module, I had to add the baud rate of 9600 to the shields. The Github repository references a specific module but my HC-05 and HC-06 do work too.

After connecting the Bluetooth module, the following screen will be shown:

vandeVelde 02 VirtualShieldHelloWorld

The “Hello world” lines are shown on the screen of your Windows 10 device. And we did not have to write a single line of C# code.

Yes, I have talked about the Lumia as a client. But in fact, every Windows 10 device can be an Arduino slave. This screendump is taken from a laptop.

If you look at the sketch, the Text.h is your reference to the screen (on which the lines are written).

I noticed that the Virtual Shields for Arduino library has some issues, I got a lot of warnings regarding colors. But they are pretty harmless:

Hello world warnings

So what else can we do?

We can access these sensors (info taken from github):

  • Accelerometer
  • Compass
  • Geolocator (GPS)
  • Gyrometer
  • Light Sensor
  • Orientation

We can access these capabilities (info taken from github):

  • Camera
  • Device info (name, date/time/timezone, os)
  • Email (initiation)
  • Microphone
  • Notifications (Tile/Toast)
  • Screen (Text, Images, Audio/Video, Rectangles, Buttons, Touchscreen)
  • Sms (initiation)
  • Speech to Text and Speech Recognition
    • can receive table data from previous web search
  • Vibration
  • Web (Get and Post with result parsing)
    • XPath
    • JSON
    • Simple text
    • Regular expressions
    • Table iteration (keys/values saved)

Nice 🙂

Microsoft has provided a lot of examples:

  • Basic-LightSensor
  • Basic-Media-Audio
  • Basic-Media-Video
  • Basic-Media
  • Basic-Microphone
  • Basic-Orientation
  • Basic-SMS
  • Basic-Speech-To-Text
  • Basic-Text-Colored
  • Basic-Text-to-Speech
  • Basic-Text
  • Basic-Vibration
  • Basic-Web
  • ControlPins
  • HelloWorld-Button-Blocking
  • HelloWorld-Speech-Eventing
  • HelloWorld
  • LedMatrix
  • MathDoorLock
  • PictureTheWeather
  • RGBLED-Draw
  • StartingTemplate-Simple
  • WeatherIndicator

All these examples provide a good introduction towards Virtual Shields. But check every example because some of them are having issues. The weather service accessed in the GPS examples only provides data about locations in the USA. So these GPS examples crash in Europe (and other non-USA regions). Other examples use specific modules (for light effects etc.) although some can be fixed using comparable modules with the same pin layout.

A more interesting example is controlling a doorlock (i use a KY-019 relay):

#include <ArduinoJson.h>
#include <VirtualShield.h>

#include <Text.h>
#include <Recognition.h>

VirtualShield shield;
Text screen = Text(shield);
Recognition recognition = Recognition(shield);

int relay = 10; // relay trigger signal - active high;

void recognitionEvent(ShieldEvent* shieldEvent)
  if (shieldEvent->resultId < 0) {
    screen.printAt(1, "Unknown error-please refresh");

  screen.printAt(1, "Heard:");
  if (shieldEvent->resultId == 1) {
    screen.printAt(2, "yes");
    digitalWrite (relay, HIGH); // relay conduction;
  } else if (shieldEvent->resultId == 2) {
    screen.printAt(2, "no");
    digitalWrite (relay, LOW); // relay switch turned off;
  } else if (shieldEvent->resultId == 0) {
    screen.printAt(2, "(nothing)");

  screen.printAt(4, "refresh to try again");

void refresh(ShieldEvent* shieldEvent)
  screen.printAt(1, "Say Yes or No");
  recognition.listenFor("yes=yes,sure;no=no,nah", true);

void setup()
  pinMode (relay, OUTPUT);

void loop()

Here you can see the how both the virtual screen shield and the virtual voice recognition shield are used. In the normal loop, the shields are checked. And whenever a predefined word is recognized, it is checked so we can act on it. So if we say “Yes” or “sure” or “No” or “Nah” the relais is switched. Pretty cool!

If you are into doing some exotic POCs with shields and Arduino’s, the Virtual Shields can be a real time saver. But the current release is sometimes not as stable as expected to test your stuff a few times more then normal.

If you are going to use sensors which provide multiple parameters, please check out this fix on the UPW app.