Volume 3, Issue 3 - May/June 2003
 
   
   
 

VoiceXML Conformance Testing

By Ken Rehor and Jonathan Engelsma

Continued from page 2...


  <?xml version="1.0" encoding="UTF-8"?>
  <!-- @ Copyright 1998-2003 W3C (MIT, ERCIM, Keio), All Rights Reserved. See
http://www.w3.org/Consortium/Legal/. @ -->  <vxml xmlns="http://www.w3.org/2001/vxml"
xmlns:conf="http://www.w3.org/2002/vxml-conformance" version="2.0">

  <var name="count" expr="1"/>

  <catch>
    <conf:fail expr="'event: ' + _event"/>
  </catch>

  <form id="exampleForm">
    <conf:grammar utterance="alpha">
      <conf:key name="x">
         <conf:key name="a" value="valueA"/>
        <conf:key name="b" value="valueB"/>
       </conf:key>
    <conf:key name="y" value="valueY"/>
  <conf:key name="z" value="valueZ"/>
  </conf:grammar>

  <initial> <conf:speech value="alpha"/>   </initial>

  <block>
     <if cond="typeof x == 'object' && x.a == 'valueA' && x.b == 'valueB'">
     <conf:pass/>
   <else/>
       <conf:fail/>
    </if>
  </block>

  <field name="x"/>
</form>
</vxml>

The resulting VoiceXML code is:


<?xml version="1.0"?>
<vxml xmlns="http://www.w3.org/2001/vxml" version="2.0">
    <var name="count" expr="1" />

    <catch>
      <prompt>fail</prompt>

      <log>failure expression:
      <value expr="'event: ' + _event" />
      </log>

      <prompt>
        <value expr="'event: ' + _event" />
      </prompt>

    <exit />
</catch>

<form id="exampleForm">
  <grammar type="application/srgs+xml" root="CityNameid2917879" version="1.0">
    <rule id="CityNameid2917879" scope="public">
      <one-of>
        <item>chicago
        <tag>var x=new Object(); x.a='valueA'; x.b='valueB'; var y='valueY'; var
        z='valueZ';</tag>
        </item>
      </one-of>
     </rule>
  </grammar>

<initial>
    <prompt count="1">Say 'Chicago'.</prompt>

    <prompt count="2">Say 'Chicago' again.</prompt>

    <prompt count="3">Say 'Chicago' one more time.</prompt>
</initial>

<block>
  <if cond="typeof x == 'object' && x.a == 'valueA' && x.b == 'valueB'">
    <prompt>pass</prompt>

    <exit />

    <else />

    <prompt>fail</prompt>

    <exit />
  </if>
</block>

    <field name="x" />
   </form>
</vxml>

Some of the conformance tests require test scripts to be hosted on application servers (i.e. consider a test assertion involving the VoiceXML fetchtimeout property). A simple web-server agnostic XML markup language called IRCGI was developed for this purpose. The server-side scripts can be written in IRCGI and transformed into specific server-side scripts (i.e. Perl, JSP, etc.).

Interim Solution for Differences in Semantic Interpretation Tags
The W3C Semantic Interpretation for Speech Recognition has been published as a working draft. However, many vendors have not yet supported it in their implementations. This introduces some difficulties in that some of the conformance tests necessarily involve semantic interpretation tags. To enable testing on a wide variety of platforms now, the conformance committee has been encouraging vendors to supply versions of the test style sheet that generate SI tags compatible with their shipping products. So far, a couple of key vendors (IBM and Speechworks) have responded to this request and provided style sheets. Several more vendors are currently working to produce their versions of the style sheet. We also support the original ECMAScript tags generated by the original W3C style sheet. We encourage vendors whose semantic interpretation tags are not compatible with the ECMAScript tags to submit style sheets to be included in the conformance test harness. When the W3C SISR specification is ratified and vendors begin to implement it, we will remove these vendor customizations, as each vendor will be expected to conform to the standard.

Test Suite Execution
To facilitate and encourage conformance testing, the conformance committee has implemented a testing harness that is now available to Forum member companies for download. The testing harness is a series of JSP pages that allow a developer to manually run specific conformance tests to determine whether or not the platform under test passes the test. In addition, the test harness implements a web-based view that allows the developer to browse assertion tests, viewing the original test scripts, the generated VoiceXML code, as well as various test dependencies.

Accessing the Test Suite
The test suite is made available in two forms: online execution directly from the VoiceXML Forum's website, and as a downloadable package for execution in your local environment. VoiceXML Forum members will not only have access to the tests, but also the testing harness itself.

A preview version of the test suite is available now:

  • To run locally: download the zip file from the members only area of http://www.voicexml.org/login.asp. You will find the download in: Members Home > VoiceXML Forum Conformance Committee > IR Test Suite.
  • To run online: go to http://cvs.voicexml.org:8282/ir (This preview version is not password protected; future version will require registration.)


    • Testing and Reporting Policy

    Once a platform vendor runs the tests, test results can be kept private or made public. Regardless, results must be provided in the Forum's report format so customers can easily compare. The VoiceXML Forum encourages all vendors to make their results public; if the vendor chooses to do so, the results can be included in the Forum's website.

    Beyond 'self testing,' to obtain VoiceXML Forum certification, a vendor must go through the certification testing process with the Forum's testing laboratory. This process is under development and will be available shortly. .

    What's Next?
    Beyond the VoiceXML 2.0 test suite, the Conformance Committee plans to include support for the W3C Speech Recognition Grammar Specification (SRGS) and Speech Synthesis Markup Language (SSML). Other W3C languages required to build portable speech applications will be added as these standards become available.

    We also plan to develop a program for application certification and development tool certification.

    Get Involved!

    Everyone is welcome to provide feedback on the Conformance Program; VoiceXML Forum members are welcome to join the Conformance Committee. You can:

  • Provide public feedback to the mailing list
  • Provide modified / corrected test scripts to committee
  • Submit new assertions for areas not currently covered
  • Design and submit new test scripts to committee
    •

    If you're an application developer, ask your vendors for their detailed test results, and require your vendors to be certified!

    References
    1. The VoiceXML 2.0 Implementation Report (http://www.w3.org/Voice/2003/ir/voicexml20-ir.html). Matt Oshry, et. al. World Wide Web Consortium. April 7, 2003.
    2. Voice Extensible Markup Language (VoiceXML) Version 2.0. (http://www.w3.org/TR/2003/CR-voicexml20-20030220/). Candidate Recommendation. W3C. February 20, 2003.
    3. Semantic Interpretation for Speech Recognition. (http://www.w3.org/TR/semantic-interpretation/). Working Draft. April 1, 2003.

    back to the top

     

    Copyright © 2001-2003 VoiceXML Forum. All rights reserved.
    The VoiceXML Forum is a program of the
    IEEE Industry Standards and Technology Organization (IEEE-ISTO).