Building
the VoiceXML Forum Certification Program
(Continued
from Part 1)
Getting
Certified
We
envision a certification program that has the steps
below that give the Vendor opportunities to explore
their product’s conformance, and then commit to certification,
and ultimately use that to market advantage.
Informal
Testing. With this first step, the Vendor obtains
test suite or access to test suite to prepare for certification.
The CA will make the test suite available to vendors
and others as appropriate; there may be a fee involved.
By getting access to the test suite in advance, the
Vendor will be able to run the tests, and understand
the outcomes before entering into the next phase of
certification.
Implementation
Conformance Statement. This is the first formal
phase of the certification process. The Vendor must
complete and submit the implementation conformance statement
that will include: definition (configuration) of what
platform will be tested, including optional features
to be claimed, and agreement to policies and legal documents
for certification. This is an important reference for
the conformance testing because the configuration being
certified will constrain the extent of the resulting
certification label once the conformance testing is
successfully passed. For example, if a Vendor’s VoiceXML
Platform were deployed in a Linux operating environment
and certified, then if the Vendor wanted to also claim
certification for a platform in a Windows 2000 operating
environment, then there may be a requirement to go through
conformance test again since the operating environment
is different. The fee to enter the certification process
would be paid to the CA at this time, and a time clock
for when the certification process must be completed
begins. This document would be confidential between
Vendor and CA; the VoiceXML Forum would not know that
a Vendor was going to undergo certification or even
interested in certification. Only at later stages,
and at the discretion of the Vendor, would the VoiceXML
Forum be told of any certification information.
The
implementation conformance statement is an important
reference for the conformance testing because the configuration
being certified will constrain the extent of the resulting
certification label once the conformance testing is
successfully passed. For example, if a Vendor’s VoiceXML
Platform were deployed in a Linux operating environment
and certified, then if the Vendor wanted to also claim
certification for a platform in a Windows 2000 operating
environment, then there may be a requirement to go through
conformance test again since the operating environment
is different; i.e., the certification would not automatically
extend. The fee to enter the certification process
would be paid to the CA at this time.
Formal
Testing. The Vendor next tests their product against
reference web server for the range for tests that are
required based on their implementation conformance statement.
Each test would have one of three outcomes: PASS,
FAIL or UNTESTED. If a test outcome
is FAIL/UNTESTED does not automatically imply that a
Vendor has failed the conformance testing because such
an outcome could become relabeled as PASS if there is
a known/discovered Test Suite Deficiency
or Specification Deficiency. A Test Suite
Deficiency is an error in a test case; such errors would
be raised during the testing, and then resolved through
the Problem Reporting process. Since a test suite,
like any other part of software development operates
under version control, a repair to a test case would
not appear until it was included in a new version of
the test suite. A Specification Deficiency is an ambiguity
or error in the specification on which the test case
is based. Such issues would be resolved through the
body responsible for the specification, and would remain
as deficiencies until they were resolved (again usually
through a new version of the specification).
Completion
of Certification. During the formal testing, the
Vendor may fail some tests and need to make corrections.
We expect that a Vendor will be able to do a limited
amount of correction retesting included in certification
fee. Finally the results of certification (pass/fail)
are maintained in confidence between the Vendor and
the CA.
Using
the VoiceXML Forum Certification Mark. Once certification
is completed successfully, it is the Vendor's discretion
on how to proceed further. There are at least the following
three choices being considered. (1) Do nothing.
Having shown that the platform passed the certification
process may be sufficient for some internal reason.
(2) Private Disclosure. The Vendor may wish to
disclose the result to a third party, such as a buyer
as part of a purchase qualification. In this case the
CA would discloses to the other party the satisfactory
completion of certification process. (3)
Public
Disclosure. In this case the CA would disclose to
the VoiceXML Forum that the Vendor’s processor has satisfactorily
completed the certification process. Consequently the
Vendor could use their certification mark on their marketing
materials, and the VoiceXML Forum could publicize the
Vendor’s certification. For private or public disclosure
the vendor will be required to sign the licensing agreement
for use of marks (this does not imply certification,
but how to behave if you are certified). The certifications
will have a valid lifetime (such as one year) with possible
renewals handled through the CA.
For
the Future
There
are a lot more details to this task than revealed in
this short article. For example, detailed processes
for how to handle changes to a certified platform, processes
for handling the problem reporting, and so on. A branding
program needs to be developed so that certified systems
can be clearly represented and identified. Next phases
of the Certification Program will include test suites
for development tools and application testing. This
information will be presented in future articles and
on the VoiceXML Forum's website as we expand these programs.
A
strong conformance program will help the VoiceXML market
grow through interoperable platforms, tools and applications.
back
to the top
Copyright
© 2001 VoiceXML Forum. All rights reserved.
The VoiceXML Forum is a program of the
IEEE
Industry Standards and Technology Organization
(IEEE-ISTO).
|
