Get off my grass

So, about that Raspberry PI project

 The minimally viable solution will capture my steering wheel controls and forward them, via Wifi to my Android phone.  The android phone will translate the keys into key events affecting my media applications to allow me to change songs, skip and any other media controls that make sense.

What I am building is excessive for the nature of the solution, but this solution is meant to teach me more about the Raspberry Pi (RPI) and the technologies.  And, there are other solutions I would like to add if I can achieve the above.  Some other considerations:

• Attach a screen.  There are some 1 DIN DVD screens that support video input that could be driven by the RPI
• Screen forwarding so that I can view maps on the display
• Push any notable alerts to the display
• Via Bluetooth communicate with a Bluetooth enable OBD2 scan tool allowing me to forward car diagnostic information to the display (sure, eye candy, but isn’t a lot of this?)

So, where am I?  Decisions, decisions, decisions

• I’ve purchased an RPI, loaded it up and am able to communicate with it remotely
• I’ve selected NodeJs as my primary runtime environment.  At some later point provide some of the thinking about this choice, but the simple reasons are: given the common language choices on the RPI, I like JavaScript the most (though I prefer static languages over dynamic).  The problem is very event driven, which fits Node well.  And, Node, very quickly supports multiple forms of communication.  A built in web server is a plus as I will use a SPA like application for my UI, once I have one.
• Communication between the RPI and Andriod will be MQTT.  Again, the nature of the protocol fits the messy communication environment within the local network.
• The Andriod device will be the Wifi source so that it can also be a hotspot.
• Android development will be the traditional Android Java development structure.

My next challenge was to understand how to get the steering wheel switch information from the vehicle.  I have a 2004, Sierra 1500.  I was having difficulty getting data online regarding the mechanism for passing the data.  Choices included direct switch data, voltage levels for different switches, a CAN network or a new network technology.

So, I pulled out my 30 year old scope that hasn’t been powered on for 15 years.  (I tried to give it away and never found somebody that could use it)  It still powered up, though it seems way out of calibration and seems to have a lot of noise coming off the leads.  In any case, I pulled out the radio and analyzed the inputs.  Found the CAN network fairly quickly and pressing the buttons clearly triggered the sending of packets.  (Edit: doing some more research, this is more likely to be a SAE J1850 network)

Next step, how to talk to the network.  There are a variety of people already doing this.  I found a bunch of project in various stages of working order.  I picked the one that looked the most complete and reviewed some of the challenges.  Before I order the parts and build it, I want to understand it.  So I am currently working my way through the data sheets.  Once I think I understand how it works and why the circuit is structured the way it is, I will get the parts and build it on a bread board.

In parallel, I’m building the basic shell of the Node and Android applications.  I have MQTT working between the devices (getting it working on Android was more complicated than expected).  I have also reverse engineered some of the messages from my favorite music and podcast players.  I’m slowly building out the command and announcement message sets that will be triggered once I can talk to the truck.

That’s the current status.  There are 3-5 entries I could make just on some of the material above.  If I get motivated, I might do so.

I haven't touched this blog in nearly four years, but I think its worth trying to post some more material.

When I originally created this blog, my goal was to document some of the VoiceXML experience I had gained building and supporting Voiyager. I was getting a very rare opportunity to see how multiple VoiceXML engines actually behaved. I did not think there are many developers doing so. The only ones I would expect to see it were other vendors that were writing multiple platform tools. And, none of them seemed to be saying much. Also, they had a different problem set. They wanted to generate code that was as portable as possible. I was stuck making our tool working with whatever application we were given used. Emulation, can be a bitch.

I originally intended this blog to document the different VoiceXML application patterns I found. There were various GUI approaches and code approaches. But, there were only a few application/architecture patterns. I did find a favourite, involving VoiceXML code generation from templates and a heavy reliance on Javascript. If I were ever in a spot where I was involved in building a large complicated VoiceXML application, I would probably follow a similar pattern. It offered easier dialog reuse (single and multiple input). However, as I had less time for Voiyager and the blog, the urgency went away to document it went away. I also wasn't seeing a lot of higher end development going on in the VoiceXML space, so I felt I was just writing into the void.

I am very proud of the work we did on Voiyager. It is a rather interesting tool and I would have loved to have taken it even farther. We had a lot of great and viable ideas. But, I started managing a larger team and more products, so my ability to focus on Voiyager went down as others took over the reins. And, almost two years ago, I changed companies and now work in a different space.

So, why reawaken the blog. Probably craziness rooting itself deeper in my head. But, the trigger was this year's Desert Code Camp(DCC). For a variety of reasons, I took a different track and looked for technologies that I thought would be more fun than career focused. When I was in school, I originally wanted to be a firmware developer. I wanted to connect the virtual and real worlds...controlling stuff and so on. That never happened. Over the years I have built all sorts of home automation projects. Last year I even built a CNC cutter. So, at this year's DCC I focused on the IOT track, originally to learn more about Arduino and similar devices. I walked away with the surprise at how much the pricing had dropped and how powerful the devices had become. I was sold.

Fast forward a few weeks. I have my first Raspberry Pi. I've picked some programming approaches and a few ideas for projects. At the top of the overly ambitious list is an automative project. The top goal is just routing steering wheel controls to my phone for music and podcasts. I have a long commute. I listen to a lot of material. Reaching over to touch the phone is less than desirable. I could move the media burden to the Pi, but the phone is actually a better tool and less synchronization hassles when a large number of podcasts and my positions within them correct (some are very long). There are other features I want to leverage including screen or content forwarding from the phone (maps, hotspot), but this will be one step at a time. There are other projects, so if this one peters out, I've already got some others to explore.

So, this blog is going to be a bit of a public diary of some of the project phases. I will also post some technical details on some of the implementation parts. In particular, areas where I don't find quick answers on the web.

Then again, this could be the last post for another four years ;-)

Out of disk space errors running Windows 7 back up ?

This one took a bit of searching to find.  In short, the System Reserved partition, which is used for boot up didn't have enough space to create an image.  In general, the typical 100M partition needs about 40M free.

I mounted the volume with disk manager, but didn't see anything that didn't belong or that accounted for the space consumption.  With some web searching, I was able to figure out that the file system journal may be at fault.  To clear out and remove the journal, you need to mount the partition to a drive letter.  I mounted to s.  Then run this command:

fsutil usn deletejournal /N /D s:

This cleared out most of the invisibly consumed space, which probably was being used within the System Volume Information folder.  

Once done, I was able to perform my regular backup.

VoiceXML Application Architecture: Server or Browser Biased

Introduction

VoiceXML applications, like any web application, can be written use a wide variety of technologies and frameworks.  And, like the web, applications can tend to take two forms, browser focused or server focused.  In the web world, browser side technologies involve AJAX like approaches using the HTTPRequest mechanism.  Server side solutions can take many forms, but these days tend to be one of the many MVC servlet frameworks. 

Even within these approaches, there are parts that are executed on the server and parts executed on the browsers.  It isn’t a one or the other solution, just where the emphasis or core logic is located. VoiceXML is the same.  Where you tend to place your core logic will be closely tied with your application architecture.  

In the sections below I provide an overview of server focused VoiceXML applications and then do the same for browser applications.  After that, I explore a variety of smaller topics and how the architecture choice impacts that topic.

Server Biased Applications

Server biased applications are the most common form of VoiceXML applications.  Server based applications tend to use VoiceXML as a more of telephony control protocol than as an application development language.  It tends to be popular for a variety of reasons, but the most popular center around using standard frameworks and the learning curve associated with VoiceXML.

The majority of code based (versus graphical development environments) applications I’ve seen are built with JSP pages.  Most of those are an ugly mixture of presentation and system logic.  In most cases, the projects don’t leverage logic used by other applications, such as the corporate web application.  Next in popularity would be Java servlet based architectures, often using the same frameworks and technologies used in corporate architectures, such as struts.  In a few cases, I have come across VoiceXML applications written in ASP.  The latter may start to become more popular as Microsoft’s Lync product suite grows in popularity.

Most of the server biased applications I’ve seen tend to break the pages into individual dialog states.  Basic recognition retry handling, no input and no match, are built into the page.  Data validation and transitions between dialogs is determined within the application server.  Some rich Voice User Interface (VUI) applications, move recognition retry logic to the server so that richer retry counters can be easily implemented.

Browser Biased Applications

Browser biased applications are significantly less common than server biased applications.  It should be noted that in this opinion, I’m ignoring small or otherwise trivial VoiceXML applications and considering applications of a moderate or larger size.

Browser biased applications may still user server side technologies for host logic.  For example, I have seen a few examples where the VoiceXML was primarily static and the server based code was primarily used for data access.  The resulting data was inserted into the document variable space and sent to the browser.  All logic and decisions, otherwise, are performed by with logic encapsulated within the VoiceXML document.

To access data, browser biased applications use the VoiceXML Data element.  The Data element came along in version 2.1 of the specification and is supported by the majority of VoiceXML platforms.  The Data element is sufficient for most host requests, but it is a vague shadow of the capabilities built into the XmlHttpRequest available in web browsers.  In particular, the Data element is synchronous.  This means they can’t be done in parallel.  Usually this just means optimizing the data requests in your host logic.  In web applications, you have the ability to update the display (HTML DOM) for nearly live interactions.  VoiceXML application DOMs are immutable so you only tend to use the Data element when you need to perform host validation or data look up.

Notable Topics

Logic Language and Variable Space

The largest impact of choosing a browser versus server biased application is the language for writing the application logic and holding variables.  For browser biased applications, there is only one choice, ECMA/JavaScript.  If you aren’t a fan or proficient in this language, this can be a strong detractor.  For server biased applications, you are typically working in Java or in the .Net world, C# or one of the other language flavors available.

I’m personally not a fan of ECMAScript due to the lack of static validation.  But, it isn’t a horrible language.  If you are going to take this route, I would strongly recommend minimizing the inline script logic and move the ECMAScript into separate files that can be easily tested outside of the VoiceXML application.  There are plenty of ECMAScript testing frameworks and if you keep your script clean (i.e. following common or standardized components) you can use other ECMAScript engines for testing and debugging.

There are rich ECMAScript libraries for manipulating XML DOM structure.  And, ECMAScript allows for rich hierarchal data structures that are often needed by complex applications.  Just remember to simplify the data structures when posting them back to the server.

Caching

Server biased applications typically cannot be cached.  You will still be able to cache to large SRGS documents along with the audio files, but the VoiceXML documents will have to be regularly fetched.  On the other hand, a pure browser biased application, and a platform with rich caching features could cache the entire application locally.  I have come across one environment where very large, very rich applications met this description.  Network traffic, in this case, was limited to data element requests and responses along with the occasional checks for updated documents.

Performance

As long as the platform has a sufficient ECMAScript and VoiceXML parsing engine, performance is usually not a significant issue in either case.  Which component becomes the bottleneck is going to typically be platform, application and environment specific.

It is worth noting that I have seen some browser biased applications that get very large.  For example, a couple 100K of ECMAScript and 30K of VoiceXML downloaded to the platform.  However, given that most of this data is cached and today’s ECMAScript engines are incredibly fast, this is usually not a problem.  VoiceXML processing can vary significantly across platforms.  In general, the older the platform and the more it had to be evolved to keep up with the emerging standards, the more likely you may see performance problems.

Data Element Platform Differences

In my write-up of platform differences, I overlooked some the differences associated with the Data element.

Data Type

The specification indicates that the data returned must be XML.  A few platforms support simple text to be returned.

Parsing of white space nodes

There isn’t a standard definition of what to do with white space nodes in the XML DOM.  These nodes can be created by simple spaces in the XML, but are more often present from end of line (EOL) characters used to nicely format the result.  Some parsers ignore the nodes, others create nodes in the DOM tree.  This can really mess up the JavaScript parsing logic as the white space nodes are often the first node in the child list.  When writing portable applications, insure that your XML is stripped of all white space prior to delivering to the browser.

 

Common VoiceXML Implementation Differences

Introduction

This article isn’t a detailed list of all of the differences I’ve found, but it does provide some explicit differences and some guidance in how to check for differences between platforms.  It is also useful to know that many IVR vendors base their VoiceXML implementation from common code bases.  Behaviors you find in one browser are likely to show up in another browser.

Common VoiceXML Enhancements

Objects

The VoiceXML standard has a built-in extension point.  The VoiceXML object element was designed to allow vendors a place where they could add their own extensions.  Usage varies significantly and most of the features exposed are very platform specific.

Transfer Element

The VoiceXML standard only supported bridge and blind transfers.  In many call centers and IVR deployments, this isn’t sufficient.  Supervised and other transfers are usually necessary and many platforms enhanced the transfer element to support alternate types.

Tone Playing

The VoiceXML standard didn’t provide a mechanism for playing tones.  These are often needed for network transfers or to drive other devices.  For traditional TDM networks, you usually just play recordings of DTMF tones.  However, in most SIP environments you can’t play tones as telephony tone data is typically carried outside of the audio band.

CTI

The VoiceXML standard nor CCXML failed to provide a sufficient mechanism for attaching data to a call or getting a unique call reference ID.  This was probably done because the availability of this information is highly environment dependent.

VoiceXML Differences

Sloppy rule enforcement

As platform vendors and behind the scenes software vendors created the initial VoiceXML browsers, many of them left holes that didn’t strictly enforce the standard.  Once those holes were created, they often weren’t fixed to avoid breaking customer applications that had been deployed. 

For example the VoiceXML VAR element, which is documented here  , can’t be used to initialize the property of an object (e.g. obj.prop1).  However, many, but not all browsers allow this behavior.

These types of problems exist in most implementations and as far as I’ve seen, are never documented.

(Embedded) Grammars

Given the way many platforms handle speech recognition, you may not be able to use embedded speech grammars within the VoiceXML.  Most platforms allow inline DTMF grammars if the platform is only running in DTMF mode.  And, a few platforms, don’t allow external grammar references (ie nearly always platforms that don’t support speech recognition)

Interaction with Speech Recognition Engine

The interface between IVR platforms and speech engines, in the past, has left a lot to be desired.  Often the platforms had to implement output and input as a turn taking exercise, which had impacts a platform’s ability to barge-in.  This problem can also manifest itself by a platform not support the VoiceXML 2.1 mark element.

Bridges to Legacy Applications

Many of the IVR platforms on the market weren’t built specifically for VoiceXML.  This means that applications often need some sort of bridge to the existing application framework.  This might mean a proprietary application to launch the VoiceXML application or it just may mean some extensions to allow a call to move back and forth between the legacy, proprietary application and the VoiceXML application.

Invalid XML

This one is a personal pain point.  I’ve come across a few platforms that support, and sometimes require, invalid XML.  I’ve seen a few flavors to date.  First, an application that didn’t need to encode common symbols, like &.  This means that you either need to preprocess the XML before using an XML editor or not use any XML tools.  The second involved incorrect encoding formats.  The document encoding would say one thing, but actually be another.  Some platforms, I believe, ignore the encoding tags.  And finally, I’ve come across cases where the platforms that don’t need the XML declaration.

Note, I’ve listed this under VoiceXML differences, but this issue often also applies to the supporting standards.

Note, I mentioned sloppy rule enforcement above.  This also applies to following the VoiceXML DTD.  I’ve come across elements that are empty, that shouldn’t be.  I’ve seen extra attributes and missing attributes.  From a developer’s perspective, many of these hassles are minor.  If you’re a tool vendor trying to support multiple platforms, it can be a bit maddening at times.

Return statements without a subdialog

I’ve come across at least one platform that ignores return statements if the call flow wasn’t currently in a subdialog.  This can hide an application bug that will show up on other platforms.

On the topic of subdialogs, I know of a case or two where I’ve seen platforms that can’t or don’t properly support the nesting of subdialog calls.

How undefined variables are handled

Some platforms generate errors, some ignore, some treat as empty strings or 0 depending on usage.

Variable posting

How variables are posted back to the application server can vary if you stray from simple usage.  As indicated above, undefined variables can yield different behavior.  And, there isn’t any standard for how objects with properties should be rendered and transmitted.  I’ve seen at least two different formats.

Built-in variables

Platform information, such as a channel number, is often provided as built-in variables.  However, the variable names and existence of them varies significantly platform to platform.

Recordings

Not all platforms follow the standard HTTP Post mechanism for transmitting recording results back to the application server.

Built-ins

Some platforms have built-in audio and/or grammars.

The Supporting Stuff

VoiceXML, by itself, is insufficient to write an application.  There are a set of defined and assumed dependencies that can cause you some difficulties.

ECMAScript

The VoiceXML specification references ECMA-262.  Not all of the browsers on the market implement an ECMAScript engine that meets this standard.  However, most either implement Rhino or SpiderMonkey  depending on whether the platform is built on Java or C++.  Both ECMAScript engines have their own extensions and few developers understand what is in the specification and what is not.  Most authors talking about ECMAScript tend to focus on the ECMAScript engines implemented in web browsers leaving developers a bit unprepared to question how they should be writing their applications.

There are a variety of other differences associated with the ECMASCript engine.  Here’s a quick list:

• Many platforms add custom ECMAScript functions to their implementation
• How undefined variables are handled
• Whether or not variables even need to be defined before use and the value of their default values

HTTP

While HTTP has been around for awhile, many of the VoiceXML browser implementations were clearly home grown and not fully functional.  Some things you may experience:

• No support for https, or support for encryption, but not certificates.
• No support for proxies.
• Session management is independent between VoiceXML engine and grammar engine.  May also apply to other features such as audio fetching, data fetching, ect.
• Caching rules can often be difficult to determine or control

Grammars

The standards for grammar development are SRGS and SISR.  These standards came later in the process and have been very slowly adopted.  It’s still more common to see proprietary implementations in the field than seeing those that follow the standards.  And, as a tool vendor, you can’t count on the formatting information provided in the documents.

SSML

Text to speech engines are even more out of sync with the standard than speech recognition engines.  You should expect to see only minimal adherence to the standards and little consistency in complex text to audio conversions.

In Conclusion

If you care about portability or are writing tools to process VoiceXML, your best chance is to get access to as many applications and platforms as you can.  Even still, you find them all.  You will only be able to minimize the issues that will occur.

The next few articles are going to discuss the different ways VoiceXML applications are built.

 

VoiceXML - Why

Introduction

The basic history of VoiceXML can be read here. From a buyer’s standpoint having a standards based solution prevents vendor lock-in.  From a vendor’s standpoint, standards have multiple problems.  If your only value point is building standards based products, you’ll be commoditized fairly quickly and profitability will be difficult. You need to be able to differentiate product and show value.  Often, there are many ways to expand or subvert a standard (the latter isn’t always intentional).  But, having a standards based product allows your sales team to create the appearance of safety in a purchase.  From a services or software solution provider, standards are great.  You can sell your service on many platforms and in some cases, the industry may become so commoditized that the products supporting them are nearly free allowing you to pocket most of the money in a deal.

As a previous application developer in the IVR application industry, I have mixed feelings regarding the standard.  In some ways, it is rich enough to build most of the content of most IVR applications.  However, it’s missing bits…sometimes important bits.  Maybe a better way to describe it is the lowest common denominator solution.  Given that nature, nearly every IVR vendor needed to extend VoiceXML functionality so that their customers could write all of the applications they needed to write for their business.  These differences mean that only a few applications are 100% portable.  But, VoiceXML is sufficiently complete that 90% or more of most applications is portable.

Impact of platform differences

As I mentioned above, few applications are 100% portable, but most applications are mostly portable.   That means with a little work, most applications can be made to run on other platforms.  As a business building or paying for an application to be built, your application investment isn’t significantly at risk as long as you retain access to talent familiar with your application that aren’t bound to the platform you’ve chosen.  If you haven’t retained the talent, you may find others to rewrite, but applications can be written in many ways and you may find that finding a resource may be more difficult than expected.

If you are a tool vendor, the differences will likely require some work and validation with each of the platforms you support.  The challenge is identifying the differences between vendors and building those differences into your framework in a clean way.  Once that work is done, the next step is to insure that the IVR vendors don’t make changes that break your product, which means regular regression testing.  This isn’t the most ideal solution, but common in any industry as most standards have some real-world issues.

In the end, the differences are a factor.  Depending on how your business managers your IVR application, it may be a minor issue or it may be no better than implementing a proprietary application.  If you aren’t sufficiently engaged in the implementation and ongoing maintenance of your application, buy VoiceXML may not save you any money nor provide you any protections that would normally be associated with following a standard.

In a later article, after I’ve discussed some of the ways VoiceXML applications can be written, I am planning to put together a discussion on the advantages and disadvantages of choosing a platform specific development language or tool.

Preview

My next article, I hope, will be useful to those writing applications for multiple platforms as I will be discussing some of the platform differences I’ve experienced over the years.

 

 

Setting up a Subversion write-thru proxy

I recently needed to figure out how to help a get a source code repository up and working.  They also have an extra challenge.  Their team is split over two locations separated by a rather poor WAN link.  For those that use subversion in this type of set, you’re probably familiar with the poor performance associated with perform file updates.  Even local updates with large source trees can take a minute or two.  Over a slow link, it can take 10s of minutes or longer. 

 

To minimize some of these pains I started exploring how to setup up a subversion with a Write-through proxy.  In this configuration you have two subversion servers.  A master and a slave.  The master lives at the primary site and the slave at the secondary site.  People at the primary site interact with the master subversion server normally.  People at the secondary site interact with the slave server in the same way they would access any subversion server.  But, at the slave server, reads and writes are handled differently.  Reads access the local server, but writes are configured in Apache to go to the master node.  This gives the remote site LAN speed updates.  Given that most subversion access is reading the repository, the user experience can be significantly improved.  To keep the repositories synchronized, the master slave is configured with a post-commit script to write the new data back to the slave.

 

In theory, it sounds great.  In my testing I ran into two problems once I got the system working. 

 

First, I found that at least the tortoise client tended to corrupt local subversion files if the network connection was unreliable during writes associated with newly added files.  This isn’t really an artifact of the proxy configuration and would be present without it.  In practice, this may not be as bad as users only add a few files at a time normally.  And, I was able to correct the corruption by directly editing the files in the .svn directly.  But, it was a rather bad user experience for a situation where I’m trying to make the lives of two groups of developers easier.

 

The second problem was a bit more serious.  Since the repository needs to synchronize after the update, I found that the database appeared to get locked during the process.  So, after a large update, on a highly constrained pipe, people at the master sites would get very low read speeds.  In other words, everybody was impacted during the synchronization time. 

 

Could these problems be worked around ?  Yes, but would generate undesirable development behaviors like only checking in at certain times of the day or just waiting as long as possible to do it.  We are also building out a Continuous Integration environment.  As part of that, I want to encourage developers to check in often.  At the moment, I’m exploring other options.  One is a commercial overlay for SVN that takes care of multisite synchronization and the other is to switch to Mercurial.  Mercurial, however, would change the workflow and tend to separate the sites.  I want the site to work together as one virtual team.

 

The instructions to set up a write through proxy are here: http://svnbook.red-bean.com/en/1.5/svn.serverconfig.httpd.html#svn.serverconfig.httpd.extra.writethruproxy

 

The documentation isn't bad, but it lacks some detail.  When you have a component improperly configured the errors and behaviors aren’t always easy to diagnose.  Here are my notes: 

1.       On both nodes

a.      Download and install Apache web server (not Tomcat)

b.      Download and install subversion: http://sourceforge.net/projects/win32svn/files/1.6.13/Setup-Subversion-1.6.13.msi/download Don’t use the customized installs as they provide their own Apache servers, which will have different configurations.

c.       Copy C:\Program Files\Subversion\bin\*.so to C:\Program Files\Apache Software Foundation\Apache2.2\modules

d.      Create C:\Repository (or some other directory)

e.      Initialize repository by running the following subversion command: svn create C:\Repository

f.        Add the following lines to httpd.conf

LoadModule dav_module         modules/mod_dav.so

LoadModule dav_svn_module     modules/mod_dav_svn.so

2.       On Slave

a.       Httpd.conf - add

<Location /svn>

  DAV svn

  SVNPath c:/Repository

  SVNMasterURI http://MasterMachine/svn

</Location>

 

<Location /svnproxy>

  DAV svn

  SVNPath c:/Repository

 

  Order deny,allow

  Deny from all

  Allow from IPAddressOfMaster

</Location>

b.      Httpd.conf - uncomment these lines

LoadModule proxy_module modules/mod_proxy.so

LoadModule proxy_http_module modules/mod_proxy_http.so

LoadModule rewrite_module modules/mod_rewrite.so

c.       Create a file with a few blank lines with the name: C:\Repository\hooks\pre-revprop-change.cmd

3.       On Master

Httpd.conf - add

<Location /svn>

  DAV svn

  SVNPath c:/Repository

</Location>

4.       On both (re)start Apache server

5.       Validate you can see both repositories with Tortoise or some other tool

6.       Optionally load master's repository data.

7.       Run the following svn commands:

a.      svnsync init http://SlaveMachine/svnproxy http://MasterMachine/svn

b.      svnsync sync http://SlaveMachine/svnproxy

8.       On the master machine, edit the post-commit hook script and add the line: 

svnsync sync http://SlaveMachine/svnproxy

 

Application Complexity

Introduction

I’m not a one-size-fits-all guy.  Not unless that size is a tent.  Evaluating solutions to a problem should take the size of the problem into account.  Reusing what you have reduces cost and maintenance and maybe even initial cost.  Internal development standards can also come into play.

When I read technology books, especially those involving a process, I’m always frustrated that the author fails to explain the external factors that aid and oppose a given solution.  Starting those conversations are important because they lead to techniques for working with those factors and where the proposed process will fail.

My background is in mid-size to enterprise (large)-size applications.  However, that doesn’t tell you very much.  The rest of this article describes a couple of different ways to look at the size of an application.  The terminology will be used in future articles when discussing the advantages and disadvantages of different application architectures.

Dialog State Count

One of the easier ways to measure an application‘s size is the number of unique dialog states within an application.  A dialog state typically consists of a prompt, input handling, including errors, and potentially result playback.

The easier aspect to this measure is when the application specification is done.  Most specifications will identify states or at least input points allowing them to be enumerated.  Determining state counts prior to writing a specification can be difficult and usually requires comparison to other applications.  In most cases, guessing on size beyond 5 or so gradients is very difficult. 

Size Terminology:

• Small – Under 50 dialog states
• Medium – 50-150 dialog states
• Large – Great than 150 dialog states

Your average banking application, without the addition of canned features and maintenance paths tends to be in the 50-150 dialog state size.  Insurance applications, due to the complexity of accounts and functionality, can easily exceed 150 dialog states.  I’ve seen numerous financial industry applications, beyond basic banking features, exceed 350 dialog states.  Often these sizes are reached by having multiple customer bases, such as business versus consumer customers, each requiring unique features and therefore unique call flows.

Dialog State Complexity

Not all dialog states are created equally, even in the same application.  There are a lot of factors that affect dialog state complexity.  Here are just a few:

• Consistent versus context sensitive prompting – In other words, are the prompts built from a static list of files or dynamic based on earlier call behavior, the caller’s profile or other factors that will have to be built into the dialog.
• Simple versus Rich output – A simple output may just be an account balance.  Rich output could involve transaction histories; each element of which could contain attributes that lead to 100s or 1000s of playback permutations.  Even simple variations can add a lot of complexity.  For example, account balances there are positive are usually presented differently than accounts that are negative.  It isn’t just a matter of playing a prompt and a number.
• Simple versus rich data playback – Playing back dates, times, numbers and spelling usually require an audio clip collection of a few hundred sound clips.  However, to make playback sound more natural, I’ve worked with collections that exceed a few thousand just for data playback.  When properly recorded and assembled, the data often sounds like a custom recording and not a collection of clips.
• Single versus multiple languages – The number of languages may or may not add a lot of complexity depending on the nature of the application and application development framework or toolset.  In many cases, just using a different audio base path for prompts and language specific engine for data is sufficient.  However, there are times when the order of playback can vary due to the language.

Many of the items above can be rendered as trivial tasks if the application framework or toolset implements the features.  Many of the features above can be incredibly painful to implement in a frameworks that aren’t designed to support them.

Task Complexity

How dialogs are strong together can have a significant impact on an application.  Many frameworks provide features for simple menu navigation.  Only a few frameworks support features that make it easier to manage complex tasks.

• Simple versus complex error handing – Single state error handling that just contains a few escalating prompts may be easy to manage.  Multiple error counters that involve the current state, current task and global error counters can add a lot more complexity to error logic. 
• Profile based prompting (personalization is one form) – The dialog flow changes based on information the application knows about the caller.  This may be as simple as skipping questions that aren’t relevant because they are already known.

Business Logic

Ideally, a large amount of your business logic should be pushed into middleware outside of your VoiceXML application.  However, far too often I see applications with a significant amount of business logic within them.  If your application communicates directly with a database or a screen-scrape emulator (yes…they still exist), you application probably has too much business logic.

Host Interface

The nature of a host interface can have a significant impact on your deployment if your chosen application framework doesn’t fully support it.  For example, many of the graphically based tools have limited interfaces that are easy to use.  Many vendors like to discuss web services as a cure-all, but the majority of implementations I’ve seen don’t support complex return results via a web service call.  In most enterprise applications, 100s of attributes are returned with an account look-up call.

A pet-peeve while I’m on the topic of host interfaces.  I’m disappointed by the number of applications written that don’t handle slow hosts properly.  If an interaction is going to take time, your application should play a suitable please wait message.  While VoiceXML does support a fetch audio clip that can be played while waiting for a response, it isn’t always a good fit for the application flow.  Properly thread your requests so that you can play complete waiting messages only when a request is taking too long.  In addition, you’ll gain the ability to time out in a reasonable period of time.  120 seconds of waiting all to be followed by a transfer to the operator is really annoying…though not as annoying as 120 seconds of silence…

DTMF Only Applications versus Speech Recognition Applications

Speech recognition can add a significant amount of complexity to an application.  And, given that speech recognition licenses are typically expensive and coupled with a significant amount of required specialty services, a lot of other applications features tend to be added into the project.  In theory, these experiences are richer and more pleasing for the caller leading to a higher appreciation.  Sometimes it is and sometimes it isn’t.  It tends to depend more on why people are calling you and how effectively they can accomplish their goals.  In many ways, the VUI (Voice User Interface) for your IVR is a lot like the GUI (Graphical User Interface) for your web site.   The price and results varies a lot from the low end to the high end.

I could write another series of articles on my opinions of various VUIs and underlying techniques, but to stay focused, here are a series of factors that boost an application’s complexity when adding speech recognition:

• Grammars – Another application for defining caller input.  The VoiceXML partner standards for grammars are SRGS and SISR.
• Tuning – The tuning process as well as exposing and setting application level parameters associated with recognition such as confidence levels.  On many platforms, you also set recognition engine settings via the application.
• Fallback error handling – allowing the caller to experience different paths depending on the difficulty they are experiencing. 

Other application features that affect complexity

Some of these are associated with speech applications, but they don’t require a speech recognition engine.

• Voice Profiles or Personas– Like languages, this involves an alternate set of audio clips.  However, usually prompts are shuffled so that repeated calls or reused dialog states sound more natural.
• Scale – IVR applications typically scale out.  If you go from 100 ports to 1000 ports, normally you are adding more systems.  This tends not to affect the application that much, but can be a factor in design and testing.
• Reliability – Fail-over for host and other interfaces may end up bleeding into your application logic.

In Conclusion

Yeah, there are a lot of factors that can affect an application.  So how do service companies quote applications?  Experience and guessing.  In enterprise deployments, the amount of effort developing an application may not be that large compared to the rest of the engagement.  Project Management is an art onto itself.  Customers are often their own worst enemies as they try to get what they want.  And, team size can play a big factor.  Get 10 specialists and 10 stake holders in a room and just watch them attempt to make a decision.  Put a fixed bid contract in the middle of that decision and you will usually have a complex deployment having very little to do with the application architecture and the development efforts that go into it.

 

An introduction to a suite of articles on VoiceXML

Preface

For the very small number of readers that I get to read this blog, I apologize for the lack of recent entries.  Between work and home, I haven’t invested much time.  But, that is changing.  My goal over the next few months is to release a series of articles related to VoiceXML and VoiceXML applications.

The Problem

I’ve been working with VoiceXML for almost a decade and while I’ve read a variety of books and blogs in that time, I see very little information about how VoiceXML is actually being used.  I don’t mean the type of applications being written, but the technologies used to put the solutions together.  Things the application architects and application developers learn after many deployments of different types of applications on different platforms. 

I suppose this makes sense.  Most VoiceXML authors come from two camps.  The first camp is trying to teach people how to implement a VoiceXML application.  VoiceXML 101 and 102 books and articles aren’t the places to discuss some of the real world challenges facing developers.  Most of these authors never get past the 102 course as the number of participants drops dramatically after the basics.  In the second camp are the vendors.  The vendors are attempting to sell things like platforms, tools and services.  They aren’t motivated to discuss problems that they aren’t solving as part of their offering.

One other thing to remember, VoiceXML is a niche solution.  While popular web technologies can be used to implement, the VoiceXML development space is incredibly small.  VoiceXML or VXML as a Google search term barely registers even compared to other telephony terms, like VoIP. 

Goal

My goal is to share some of what I’ve learned about VoiceXML.  Some of the topics I intend to cover in these articles:

• An overview of how I look at application complexity
• The typical implementation methods for those outside the industry
• A background on VoiceXML
• Challenges with the VoiceXML standard
• Application development tools and architectures – an architectural view of different ways VoiceXML applications are being put together today.

The last item is the focus of the articles and is the topic that I believe is being given very little attention.

After these items, there are a variety of other areas I might explore.  I’m open to suggestion.

Disclaimer

I’ve intentionally omitted my employer’s name to avoid any direct links with them and to avoid having this show up on everyone’s Google Alerts for the company.  However, the blog is tied to my Facebook page and therefore, won’t stay hidden.  These articles are my own opinions and are not reflective of my employer.  Some short web searches should reveal my employment history.  Please focus any criticisms directly on me.

My Background

In the decade I have worked with VoiceXML, the first half was part of a professional services organization.  I had a lead/architect role working with developers, project managers and customers creating both custom and packaged VoiceXML solutions.  In the latter half of the decade, I transitioned to a product development role.  One of my products is a VoiceXML application testing tool.

Intentional Omissions

This won’t be a tell-all discussion.  My goals aren’t to shame or promote any company or approach.  I am not even interested in pushing a particular type of solution.  The approaches and solutions, for the most post, will be presented in an abstract manner.  In a few cases, such as tool developers and speech vendors, it is nearly impossible to avoid mentioning them by name because of their size and placement within the industry.  If you are looking for a consumer reports style article, this set of articles isn’t for you.

 

This is a personal weblog. The opinions expressed here represent my own and not those of my employer. Nobody may take credit for my stupidity other than me.

 

Nice post on design in an Agile process: http://neverletdown.net/2010/08/choosing-a-software-design-strategy/