In cases where a person must perform some task without the use of their hands, systems are developed which allow manipulation of the environment by other means. Some of these examples include chin/tongue/nose/mouth/limb-operated controllers as well as systems controlled by speech or a laser beam interface using the person's eyes. These systems tend to be expensive and the user must be trained over a substantial period of time before becoming able to successfully operate them.
The development of a digital camera-based interaction system which performs repeated detection of the darkest part of any facial feature of the user's choice is reported. This detection provides the control information to a simple robot arm which is visually displayed to the user for feedback in the form of a virtual image. By moving an orthogonal pair of these facial features in three dimensions, the user is able to control the real robot arm to pick and place objects located within the workspace.
Microsoft's Visual C++ 4.0, a 32-bit compiler, was used with the Microsoft Foundation Classes, together with the OpenGL programming interface, enabling the display of a three-dimensional representation of a real workspace. Two extra parallel ports were installed in the experimental PC platform to interact with the digital cameras. These cameras were controlled using commands which gave access with the digital pixel value so that simple digital image processing techniques could be applied. A robot arm and its control circuitry were constructed using inexpensive integrated circuit chips, stepper motors and construction materials.
Using two digital cameras provides enough information to control a simple robot arm for basic object placement tasks. Due to the limited functionality of the real robot arm system, only small, lightweight objects are able to be manipulated, but the overall design of the interfacing system proves successful. By increasing the size and quality of the real robot arm construction, a very effective system of real-world control can be made available to those who need to do work without the use of their hands and without the expense and lengthy training periods required in the past.