Project Definition

Introduction

The virtual she-butler comes to life through project jaNET, a platform ready to host multiple creative and innovative pieces of code and hardware concerning domestic everyday tasks.
jaNET provides a built in framework to fulfil most of our daily information manners (like e-mail notification, weather conditions, appliances control, schedulers etc) but it can also be extend by wrapping 3rd party applications, processes or scripts in order to become a more flexible and smart system!

Framework

Built-in functions and API commands, are described below and it used for instruction set customization.

Functions

User and Greetings
  • %user% %whoami% - return user login
  • %daypart% %partofday% - return the current part of the day (morning, noon, evening, afternoon, night, midnight)
  • %salute% - return human greetings e.g.: good morning, good evening and good afternoon

    Check-in/Check-out
  • %usercheckin% %checkin% - change user status to true
  • %usercheckout% %checkout% - change user status to false
  • %whereami% - return the user status (absent/present)

    Weather (retrieve information from yahoo web service http://developer.yahoo.com/weather)
  • %todayday% - return current day
  • %todayconditions% - return current day conditions http://developer.yahoo.com/weather/#codes
  • %todaylow% - return current day low temperature
  • %todayhigh% - return current day high temperature
  • %tomorrowday% - return the day after
  • %tomorrowconditions% - return the day after conditions
  • %tomorrowlow% - return day after low temperature
  • %tomorrowhigh% - return day after high temperature

    Localization
  • %day% - return system day i.e. Sunday
  • %date% - return current system date i.e. November 5
  • %calendarday% - return system day i.e. 5
  • %calendarmonth% - return system month i.e. 11
  • %calendaryear% - return system year i.e. 2013
  • %calendardate% - return system date in d/m/yyyy format
  • %time% - return system time
  • %time24% - return system time in 24 format (hh:mm)
  • %hour% - return system hour i.e. 17
  • %minute% - return system minute

    Net
  • %pop3count% - pop3 unread message count
  • %gmailcount% - gmail unread message count
  • %gmailheaders% %gmailreader% %gheaders% %greader% - gmail headers, sender information and subject

    System
  • %mute% - disable speech synthesis
  • %unmute% - enable speech synthesis
  • %cls% %clear% - clearing the console
  • %quit% %exit% - terminate application
  • %about% %copyright% - return framework version and copyright information

API

1. Instruction Sets & Events
    1.1 Add New Instruction Set
        + judo inset add <lock>[ID]</lock> <lock>[Action]</lock>
        + judo inset new
<lock>[ID]</lock> <lock>[Action]</lock>
        + judo inset set
<lock>[ID]</lock> <lock>[Action]</lock>
        + judo inset setup
<lock>[ID]</lock> <lock>[Action]</lock]
        + judo inset add
[ID] <lock>[Action]</lock> `[Category]` `[Header]` `[Short Description]` `[Long Description]` `[Thumbnail Url]`
        + judo inset new [ID] <lock>[Action]</lock> `[Category]` `[Header]` `[Short Description]` `[Long Description]` `[Thumbnail Url]`
        + judo inset set [ID] <lock>[Action]</lock> `[Category]` `[Header]` `[Short Description]` `[Long Description]` `
[Thumbnail Url]`
        +
judo inset setup [ID] <lock>[Action]</lock> `[Category]` `[Header]` `[Short Description]` `[Long Description]` `[Thumbnail Url]`
    1.2 Remove Instruction Set
        + judo inset remove <lock>[ID]</lock>
        + judo inset delete <lock>[ID]</lock>
        + judo inset del <lock>[ID]</lock>
        + judo inset kill <lock>[ID]</lock>

    1.3 List Items
        + judo inset list
    1.4 Add New Event Handler
        + judo event add [ID] <lock>[Action]</lock>
        + judo event new [ID] <lock>[Action]</lock>
        + judo event set [ID] <lock>[Action]</lock>
        + judo event setup [ID] <lock>[Action]</lock>

    1.5 Remove Event Handler
        + judo event remove [ID]
        + judo event delete [ID]
        + judo event del [ID]
        + judo event kill [ID]

    1.6 Delay Between Actions
        + judo sleep [timeout in ms]
        + judo timer [timeout in ms]

    1.7 List Items
        + judo event list

2. Mail
    2.1 Smtp Settings
        + judo smtp add [host] [username] [password] [port] [ssl]
        + judo smtp setup [host] [username] [password] [port] [ssl]
        + judo smtp set [host] [username] [password] [port] [ssl]
        + judo smtp settings

    2.2 Pop3 Settings
        + judo pop3 add [host] [username] [password] [port] [ssl]
        + judo pop3 setup [host] [username] [password] [port] [ssl]
        + judo pop3 set [host] [username] [password] [port] [ssl]
        + judo pop3 settings

    2.3 Gmail Settings
        + judo gmail add [username] [password]
        + judo gmail setup [username] [password]
        + judo gmail set [username] [password]
        + judo gmail settings

    2.4 Send
        + judo mail send [from address] [to address] `[subject]` `[message]`

3. SMS
    3.1 Settings
        + judo sms add [api id] [username] [password]
        + judo sms setup [api id] [username] [password]
        + judo sms set [api id] [username] [password]
        + judo sms setings

    3.2 Send
        + judo sms send [phone number] [message]

4. Scheduler
    4.1 New Schedule
        + judo schedule add [name] [{single day: e.g.Monday} {d/m/yyyy} {daily} {workdays} {weekend}] [hh:mm] [{instruction set} {verbal notification}]
        + judo schedule new [name] [{single day: e.g.Monday} {d/m/yyyy} {daily} {workdays} {weekend}] [hh:mm] [{instruction set} {verbal notification}]
        + judo schedule set [name] [{single day: e.g.Monday} {d/m/yyyy} {daily} {workdays} {weekend}] [hh:mm] [{instruction set} {verbal notification}]
        + judo schedule setup [name] [{single day: e.g.Monday} {d/m/yyyy} {daily} {workdays} {weekend}] [hh:mm] [{instruction set} {verbal notification}]
        + judo schedule add [name] [{repeat} {timer} {interval}] [interval in ms] [{instruction set} {verbal notification}]
        + judo schedule new [name] [{repeat} {timer} {interval}] [interval in ms] [instruction set} {verbal notification}]
        + judo schedule set [name] [{repeat} {timer} {interval}] [interval in ms] [instruction set} {verbal notification}]
        + judo schedule setup [name] [{repeat} {timer} {interval}] [interval in ms] [instruction set} {verbal notification}]

    4.2 Remove Schedule
        + judo schedule remove [name]
        + judo schedule delete [name]
        + judo schedule del [name]

    4.3 Disable Schedule
        + judo schedule disable [name]
        + judo schedule deactivate [name]
        + judo schedule stop [name]
        + judo schedule off [name]

    4.4 Enable Schedule
        + judo schedule enable [name]
        + judo schedule activate [name]
        + judo schedule start [name]
        + judo schedule on [name]

    4.5 Remove All Schedules
        + judo schedule remove-all
        + judo schedule delete-all
        + judo schedule del-all
        + judo schedule cleanup
        + judo schedule clear
        + judo schedule empty

    4.6 Disable All Schedules
        + judo schedule disable-all
        + judo schedule deactivate-all
        + judo schedule stop-all
        + judo schedule off-all

    4.7 Enable All Schedules
        + judo schedule enable-all
        + judo schedule activate-all
        + judo schedule start-all
        + judo schedule on-all

    4.8 List Actives [ Names ]
        + judo schedule active
        + judo schedule actives
        + judo schedule active-list
        + judo schedule active-ls
        + judo schedule list-actives
        + judo schedule ls-actives

    4.9 List Inactives [ Names ]
        + judo schedule inactive
        + judo schedule inactives
        + judo schedule inactive-list
        + judo schedule inactive-ls
        + judo schedule list-inactives
        + judo schedule ls-inactives

    4.10 List All [ Names ]
        + judo schedule names
        + judo schedule name-list
        + judo schedule name-ls
        + judo schedule list-names
        + judo schedule ls-names

    4.11 List Actives [ Details ]
        + judo schedule active-details
        + judo schedule actives-details
        + judo schedule active-list-details
        + judo schedule active-ls-details
        + judo schedule list-actives-details
        + judo schedule ls-actives-details

    4.12 List Inactives [ Details ]
        + judo schedule inactive-details
        + judo schedule inactives-details
        + judo schedule inactive-list-details
        + judo schedule inactive-ls-details
        + judo schedule list-inactives-details
        + judo schedule ls-inactives-details

    4.13 List All [ Details ]
        + judo schedule details [name (optional)]
        + judo schedule list [name (optional)]
        + judo schedule ls [name (optional)]
        + judo schedule status [name (optional)]
        + judo schedule state [name (optional)]


5. Socket Communication
    5.1 Start Service
        + judo socket start
        + judo socket enable
        + judo socket on
        + judo socket open
        + judo socket listen

    5.2 Stop Service
        + judo socket stop
        + judo socket disable
        + judo socket off
        + judo socket close

    5.3 Status
        + judo socket status
        + judo socket state


6. Web Server
    6.1 Start
        + judo server start
        + judo server enable
        + judo server on
        + judo server listen

    6.2 Stop
        + judo server stop
        + judo server disable
        + judo server off

    6.3 Create/Change Login
        + judo server login [username] [password]
        + judo server cred [username] [password]
        + judo server credentials [username] [password]

    6.4 Status
        + judo server status
        + judo server state


7. Serial Port
    7.1 Open
        + judo serial open [port (optional)]
    7.2 Close
        + judo serial close
    7.3 Send Command
        + judo serial send [command]
    7.4 Listen/Monitor
        + judo serial listen [timeout in ms (optional)]
        + judo serial monitor [timeout in ms (optional)]

8. Help
    8.1 Preview All
        + judo help
        + judo ?

    8.2 Preview Specific Category
        + judo help [help keyword]
        + judo ? [help keyword]


(*) Brackets are mandatory when place a sentence as one argument.
(**) <lock>parser protected document</lock> Lock tags used to bypass parser.
(***) Help Keywords: inset, event, mail, sms, schedule, socket, server, serial, help.

Instruction Synthesis

Command synthesis achieved through an xml description file. This file contains customizable Instruction Sets, Events and Settings.

There are two kind of InstructionSets. Those characterized by asterisk (*) wildcard and those without it. The asterisk is a pointer and indicates the execution of an external resource or a judo command in order to fetch a result or trigger an action. Notice that they cannot triggered by themselves and they will be unavailable unless they bind or referred to an InstructionSet. Any additional text inside the trigger will be produced verbally.

Lets see some examples.

Wrap a shell script to get gmail count.

    <InstructionSet id="*gmail">./sh /home/john/bin/jaNET/gmail.sh</InstructionSet>
    <
InstructionSet id="gmail">You have, *gmail, mail messages to your g-mail account.</InstructionSet>
   
When gmail command is invoked, the parser will execute the gmail.sh, fetch the result and synthesize the final output like, "You have, 2, mail messages to your g-mail account."

Notice that with ./ you can execute any script or program and retrieve the result.

judo commands can be executed as well. For example, if I want to send a message to a microcontroller, then I use the judo serial send <command> where command is what the microcontroller is programmed to receive in order to return a result.

    <InstructionSet id="*gettemp">judo serial send temp</InstructionSet>
    <
InstructionSet id="temperature">Current temperature is *gettemp.</InstructionSet>

Another example is, synthesis with framework function calls. For instance when security breach occurred, triggered by a pir sensor, it can be something like:

    <InstructionSet id="securitybreach">Security Breach @ %date% - %time24%</InstructionSet>

this will return current date, for example June 15 and time in 24 format, let's say, 12:00.


But it doesn't stop there! You can also use both methods. Probably you may want to have an InstructionSet that returns both of your gmail and pop3 account.

    <
InstructionSet id="allmail">You have, %mailcount%, mail messages, to jambel.net account, and, *gmail, mail messages to your g-mail account.</InstructionSet>

This "hybrid" mode will "ask" the framework with %mailcount% method, to retrieve the number of your pop3 incomings you have already setup to jaNET and the other part as I previously mention.

Now, what if we want to raise more than one events with one command. This can be done by have multiple Instruction Sets with same command name but different events.

    <InstructionSet id="forecast">Today %todayday% in Athens, %todayconditions%, between %todaylow% and %todayhigh% celcius degrees.</InstructionSet>
    <InstructionSet id="forecast">Tomorrow %tomorrowday% in Athens, %tomorrowconditions%, between %tomorrowlow% and %tomorrowhigh% celcius degrees.</InstructionSet>

Similar when it comes to settings and events. You can get an idea by editing the AppConfig.xml file.

jaNET can accept multiple commands delimited by semicolon.

To get an idea check out the initial sample configuration file.

Evaluation

Follow this post.

Technical Specifications

Recommended OS, Ubuntu linux 12.04 and later. Partly on Windows (jubito is fully tested in Windows) and none on Mac OS.
Development platform, monodevelop.
.NET Version: 4
Speech recognition software, Simon.
Speech synthesis engine for linux, Festival Speech Synthesizer and default speech engine for Windows.
Hardware phidget modules developed in java. Obsolete, replaced by jBox.

Resources

Ubuntu
Mono Develop
Festival Speech Synthesis
Simon Speech Recognition
Arduino Uno + various sensors in jBox
Phidgets
Java
Comments