Student Project at Kutztown University Has Amazon Alexa Talking to Extron Classroom AV Systems

Our success in completing the CT-Toolbox project is partly attributable to our Computer Science and Information Technology core courses. The user-friendliness of Extron Global Scripter and Alexa Skills Kit were a big help, as was the patient support provided by our advisors, including Extron.

Kutztown University of Pennsylvania was founded in 1866 under the name Keystone State Normal School to train teachers who would educate the workforce of an increasingly industrialized upper Berks County, 50 miles northwest of Philadelphia. In the 150 years since then, it has become a leading higher education institution in the region, offering 130 undergraduate programs and 31 graduate programs to a student body of approximately 8,000. Three of these students and their faculty advisors recently completed a project dubbed “CT-Toolbox”, which interfaces Amazon Alexa with Extron control to allow voice control of the college’s Extron classroom AV systems.

Challenges

When considering possibilities for a senior project, students Justin Harris, William “Billy” Henry, and Hannah Moyer, with majors in computer science and information technology, formed a team to create a pilot project to further the school’s strong commitment to the Americans with Disabilities Act. The Classroom Technology Toolbox, or CT-Toolbox, supports this commitment by providing a way to control classroom AV systems with voice commands. The project was suggested by computer science and information technology professor Donna DeMarco, who is also co-founder of a company that provides online training and development solutions to businesses.

Solution

The University’s Classroom Technology Manager, Ty Schwab mentored the student team, provided hardware and software resources, and secured a classroom for project development. Ty’s predecessor Troy Weidner, who recently retired after 35 years, shared mentoring duties. Robert Kennedy, Media and Control Systems Engineer at the neighboring University of Scranton, provided insights gained from his own successful Alexa-Extron voice control project, rationing his knowledge so the students could find their own way. Extron sales and marketing managers Jason Bond and Peter Knapp helped with Extron control system drivers and Extron Global Scripter^® software programming suggestions.

Extron-Equipped Classroom AV Podium Functions as CT-Toolbox Testbed

Kutztown University has 235 rooms equipped with AV systems connected to the school network. These systems share a common design. The version used for the CT-Toolbox project consists of a podium built around a desktop PC with a local monitor and an Extron IN1608 xi IPCP Eight Input Scaling Presentation Switcher. The switcher accommodates a wide range of AV sources and displays and includes a built-in AV control processor and 50-watt stereo amplifier. Sources include a PC, document camera, and a Blu-ray player. A projector with HDMI input displays the video, while audio plays through ceiling-mounted speakers. An EBP 200 eBUS Button Panel provides source selection, projector control, and volume control. In larger rooms, switching is handled by a DTP CrossPoint 84 4K, an 8x4 Scaling Presentation Matrix Switcher that is paired with an IPCP Pro 550 Control Processor and Touchlink Pro Touchpanel.

The project got rolling with a deep dive into Amazon Alexa AWS documentation and Extron Global Scripter documentation. Development roles were assigned: Justin, team lead, took hardware, drivers, and Global Scripter controller coding; Billy took Alexa coding and AWS Cloud Computing services; Hanna covered UX – User Experience design and testing.

Alexa Skills Kit and Extron Global Scripter Are Key Tools Used to Develop CT-Toolbox

The architecture diagram shown in this article’s sidebar to the right illustrates two software services, Alexa Skill and the Toolbox Client, implementing the voice control process. The project team used the Alexa Skills Kit – ASK – to develop custom Alexa skills to control the AV podium hardware. The user speaks a command using the trigger word “Toolbox” into an Echo™ device. Running in the cloud on Amazon Web Services – AWS, Alexa Skill translates the speech to a JavaScript Object Notation – JSON – data string that is passed to skill application code, called a Lambda Function. The Lambda Function parses the JSON data into the parts needed to trigger the desired hardware actions, based on a Lambda Function list of ten to twenty spoken words that a user could use to signal “intent.” The team wrote the Lambda Functions in Python, using Extron Global Scripter. The code runs on the AWS Lambda cloud service.

Amazon Cloud Communicates with Extron Control Processor

The parsed commands are delivered from the cloud to the Toolbox client via Amazon Simple Queue Service – SQS, a managed message queuing service. The Toolbox client, running Global Scripter on the AV podium’s Extron control processor, tells the control processor to output the required hardware command to the applicable device: e.g., “turn-on the projector.” The device is commanded to the required state via hardware driver code communicating via the control processor port appropriate to the device being commanded, i.e., serial RS-232, USB, Ethernet. The entire process is reversed to translate status from the commanded device into verbal confirmation from Alexa that the command was executed.

Results

In late November 2020, the CT-Toolbox student team and advisors gathered for “Demo Day,” where the students reported on their senior project results. They successfully demonstrated the feasibility of performing hands-free voice control of the school’s standard classroom AV systems. Check-out a snippet of their Demo Day video presentation in this article's sidebar at right. There are practical issues to address before any kind of campus-wide rollout: security of the cloud-based system, hosting of the Toolbox client on the network instead of on individual AV podiums, and wariness about Alexa “always listening.” However, advisor Ty Schwab says that he would definitely consider an ADA-related request from a professor to implement CT-Toolbox in a specific classroom. At the end of Demo Day, CT-Toolbox received a passing grade. All three students went on to graduate and are busy starting their careers.