Embedded Touch Screen Interface: NetBurner at your Fingertips

touch_screen_netburner

NBTouch: NetBurner at your fingertips

Embedded systems are notoriously difficult to work with for beginners (even with NetBurner’s simple and easy-to-use tools, non-engineers sometimes find it intimidating). Also, it is hard to develop good-looking and friendly user interface(UI) for an embedded device. This project will shows you how to develop a easy to use touch screen device to display web interface on a NetBurner device. This project will allow the integration of all kinds of friendly UI as it is only a web application. For ease of use, I focused on having it set up and run with no required user input.

To make such a product, the goal for the project was to meet the following expectations:

  • Touchscreen
  • Virtual keyboard
  • Minimal window manager / no desktop
  • Automatically find local NetBurner devices
  • Display the devices’ GUI

In this article, I will show you a quick demo and step-by-step how-to guide to make your application efficient, convenient, and simple by displaying your UI on a touchscreen kiosk that meets my listed expectations.

Demonstration

Flowchart for the NBTouch

Preparation

Hardware:

  1. Capacitive LCD Touch screen: One that I suggest and used for this article is
    WaveShare Electronic’s 7-inch HDMI LCD.
  2. Appropriate connections and development tools are necessary:
    • Ethernet cable
    • micro SD card and adapter
    • keyboard and mouse
  3. You will also need a Linux embedded system (I used a Raspberry Pi B+ version 1.2).

Software:

    1. Win32 Disk Imager: also works in 64-bit OS
    2. SD Formatter
    3. Raspbian OS: installation could be done using NOOBS (New Out Of Box Software) which Raspberry Pi provides, but I did not want the recovery screen(shown below) to show up on every boot-up, so I choose to flash the OS image to the SD card using the win32diskimager.

How To

Step 1. Format the SD card

Warning: This step will wipe your SD card! Make sure to back up everything you want to save before you continue.

First, you need to format your SD card. Open up SD Formatter and make sure the drive that the SD card is on is selected, then open up Options.

Set FORMAT SIZE ADJUSTMENT to On, press OK, then press Format.

format SD

Without FORMAT SIZE ADJUSTMENT, some card readers or host devices will not be able to use the full capacity of larger SD cards.

Step 2. Load OS image onto the SD card

After the SD card has been formatted, load the Raspbian OS image using Win32 Disk Imager.

win32 disk

You must locate the image file and the device you wish to load. Then pressWriteto begin the process.

Step 3. Connect hardware (keyboard, mouse, Ethernet, HDMI, USB, and the pi power supply)

connect hardware

Once you supply power, the device will turn on and open up the boot menu. First, you will need to expand the file system to utilize the full space by selecting
Expand Filesystem.

expand filesystem

Then, navigate to Internationalisation Options, select change keyboard, then pick Generic 105-key (Intl) PC unless you want another specific keyboard on the list.

105 key

Select your language by selecting Other on the bottom (the default is English UK).

configure keyboard

keybaord

Lastly, you want to set it so that it boots up to the desktop in order to utilize the GUI during development.

This is done by choosing the third option in the main menu Enable Boot to Desktop/Scratch, then  choosing Desktop Log in as user Pi.

login

Finish and reboot.

Step 5. Configure touchscreen

This step may differ if you choose to implement a different display.

After the reboot, you can start the terminal and type:
sudo nano /etc/xdg/lxsession/LXDE-pi/autostart

This command will open the autostart file with the text editor called nano.

In this file, you should find the first line to be:
@lxpanel --profile *things* ...Change this line to just be:
@lxpanel ...This ensures that the OS uses the stable default panel.

For the display I am using, the driver can be downloaded here.

On the terminal, extract the tar.gz file with the command:
sudo tar zxvf RPI2B_B_B+_USB_TOUCH_CAP_7.0_RASPBIAN.tar.gz

Navigate to the extracted folder and run the following script:
cd RPI2B_B_B+_USB_TOUCH_CAP_7.0_RASPBIAN sudo ./USB_TOUCH_CAP_7.0_RASPBIAN

After rebooting, your system will enable usage of the touchscreen.

Step 6. Download software

Next, set up the software that are needed with the following commands:

sudo apt-get update -y sudo apt-get install matchbox-keyboard chromium matchbox-window-manager unclutter -y

  • matchbox-keyboard: add support for the virtual keyboard
  • chromium: I have tried several different browsers, but I recommend chromium in order to utilize the application mode
  • matchbox-window-manager: free and open source minimalistic window manager that does not need extra configuration
  • unclutter: this app hides the mouse cursor after brief inactivity

Step 7. Set up a new user

For additional security, set up a new user account that does not have super user permission.

sudo adduser nburn

Set password to nburn as an example. If you want to change the username and password, you must also change the hard coded directory paths for the scripts that we will use to load up the application. The path is hard-coded to avoid having the user read and write at unwanted places or ask for permission to do so.

After the new user is set up, download the two scripts and settings file (readandopen.py, searchandwait.py, and xinitrc) to the new user’s directory. After downloading them into the user’s home directory, you will need to change xinitrc into .xinitrc with command:
mv /home/nburn/xinitrc /home/nburn/.xinitrc

Step 8. Disable power save

If you want to enable power saving mode for the display (blank screen), then you can skip this step.

Otherwise, you can disable the power save setting by following the below instructions.

sudo nano /etc/lightdm/lightdm.conf

This opens up the configuration file for the LightDM X display manager. Move down to:
[SeatDefaults]or search by
ctrl+win order to change the following line:
xserver-command=X

to this:
xserver-command=X -s 0 -dpms

In the above command line, -s sets the screen saver timeout in minutes; setting it to 0 will disable the screensaver, and -dpms will disable display power management services.

Now press
ctrl+
x, then
Y, and then
enter to save the changes.

Step 9. Extra Pi Aesthetic Features

This step is to not necessary for the functionality of the display, but it will make the application look better at boot time.

pi features

You can hide the logo that appears at the top-left corner.

Open the command line file with sudo nano /boot/cmdline.txt then add logo.nologo in front of the contents.

Silence boot log

If you feel that boot the log messages add too much clutter, you can try this solution. Unzip the file and replace the following contents:

  1. /etc/init.d/dphys-swapfilewith contents of:
    dphys-swapfile(1)from the zip using copy and paste
  2. /etc/init.d/fake-hwclockwith contents of:
    fake-hwclock
  3. /sbin/dphys-swapfilewith contents of:
    dphys-swapfile
  4. /lib/lsb/init-functionswith contents of:
    init-functions
  5. /lib/lsb/init-functions.d/20-left-info-blockswith contents of:
    20-left-info-blocks
  6. Lastly, add
    loglevel=1to activate change in cmdline.txt (same as Hide log section above).

Reference: https://www.raspberrypi.org/forums/viewtopic.php?f=29&t=17269&p=174897

Step 10. Overview of the system, auto start script on login

At this point, most of the basic system settings have been set up. I wrote a script (searchandwait.py) that is meant to automatically run after the user logs on (called by .bash_profile). This script searches for NetBurner devices and then either waits for the interrupt or starts the display with the default choice. Once the display is booted up, then another script(readandopen.py) automatically runs (called by .xinitrc). This will read the configured files and then open up the proper application’s website. The second part is already set up when .xinitrc is in place in step 7; the first part needs to be set up by making a new file called .bash_profile:
cd /home/nburn/ sudo less .bash_profile

Write the following inside the file:
/home/nburn/searchandwait.py startx

then press:
ctrl+
xand
ctrl+
w, and then
enter.

Step 11. Auto login

Everything is set up now! The last thing you want is to have it boot up with the user account you created. At the terminal, type:
sudo nano /etc/inittabthen find a line where it says:
1:2345:respawn:/sbin/getty 115200 tty1,and change it to:
1:2345:respawn:/sbin/getty --autologin {USERNAME} --noclear 38400 tty1

Closing

When I first started this project, I thought it would only take a day or two. I was wrong; as each milestone of the project was met, I wanted to make it more convenient, usable, and stable. For example, after I finally got everything working on the Linux desktop, I decided to change so that it does not require booting up to the Desktop. I wanted it to use minimal resources and run only core programs. Another example is when I got the system to boot off the hard coded MAC(Median Access Control) properly. I wanted it to automatically search for available devices that it can connect to, and then give the user the option to choose which device to make a connection. Another challenge was writing python scripts. Learning a new language to do things like execute shell command, do basic file operations, work with user input, and use signals and alarms had to be done in a short development cycle. It was challenging and fun. Mostly, I am proud and confident in this product; I know it will become a very useful tool. I hope you guys enjoyed my article! If you want to check out the script without downloading, then check here. If you would like to read more in depth, I will discuss some design logics below. Thanks for reading!

Advanced Reading

You might be wondering why NetBurner does not integrate the display component into their own modules? There are a couple of reasons:

Delegation

While NetBurner modules are capable of integrating stand-alone graphical environment, embedded systems should be lightweight and efficient. Loading up the graphical display driver and running it would not only consume limited flash storage, but also limited processing resources. It would make sense to have the module do what it does best, and make the stand-alone display interface serve as a generalized UI for all other devices (like how monitor can be used for different desktops).

Real-Time OS vs Preemptive multitasking

Another question is: why not just run everything in Linux? If you are looking for a real-time computing solution, Linux is not what you want to consider. All variants of Unix and other modern OS, and Linux(Raspbian in our case) support preemptive multitasking. However, Linux’s real-time scheduling is soft real-time. This means the scheduler offers certainty about relative priorities, but the kernel does not offer any guarantees on how quickly the process will be scheduled once it becomes runnable. So, if you want a real-time computing solution, you should be looking at an RTOS. NetBurner has its own OS and it can guarantee a minimum latency between when they start running as they become runnable. Therefore, it would make sense for a product with a display to use an efficient OS such as an RTOS. Another problem is the issue of communicating with and controlling hardware. With Linux, this has to be done with kernel drivers and must be split off the main application. They are difficult to write and use, therefore complicating otherwise simple applications.

Share this post

Subscribe to our Newsletter

Get monthly updates from our Learn Blog with the latest in IoT and Embedded technology news, trends, tutorial and best practices. Or just opt in for product change notifications.

Leave a Reply
Click to access the login or register cheese