See also: IRC log
<shadi> http://www.w3.org/WAI/ER/conformance/comments-20140130
Eric: Some discussion last week
... A link was sent for the minutes of the previous meeting
... We needed more discussion on the scope last week
... we need to go through major points within Disposition of comments
<shadi> http://www.w3.org/2014/03/27-eval-minutes
Eric: We could put points from discussions in
surveys which could be circulated
... We talked about version from title, scoring (needs a bit more)
... As many comments as possible will be put in the survey so we can focus our
discussions
<Detlev> fine
Eric: This will also allow others to look at the work
Shadi: Last week we debriefed from
face-to-face
... the survey idea for comments re disposition of comments is a good idea
<shadi> http://www.w3.org/TR/WCAG-EM/#step1d
<shadi> Step 1.d: Define Evaluation Methods to be Used (Optional)
Shadi: Steps requiring further thought 1d
... 1d needs to be better defined, with more explicit reference to techniques
and tools
... Email, or a survey or thinking about the questions in the survey would be
a good place to start this discussion
<shadi> http://www.w3.org/TR/WCAG-EM/#step4a
Shadi: step 4a
<shadi> Step 4.a: Check for Each Success Criterion
Shadi: we need to check back to the minutes from
the face to face - IBM comments?
... There were some gaps issues
<shadi> http://www.w3.org/TR/WCAG-EM/#step5d
Shadi: last scoring
<shadi> Step 5.d: Provide an Aggregated Score (Optional)
Shadi: this needs to be discussed further with
several groups, including WAI coordination group
... We need to make sure that all things we do are in line with the overall
goals of WAI
... Are there other topics which need more discussion?
<ericvelleman> http://www.w3.org/TR/WCAG-EM/#step1c
Eric: Step 1c
... We had a bit of trouble defining this
... Difficult to describe baseline
... need a bit more guidance
... Defining the baselines becomes very broad
... There are a number of sections to be addressed - total of 4
... should we go through the 4 sections
Shadi: Its fine that we raise issues ourselves
... define the baseline is a candidate for discussion with the WCAG group on
accessibility support - understanding doc
Kathy: Discussion in WCAG working group has had some discussions
Shadi: Kathy to be coordination point between
WCAG group and EVAL re baseline
... We need to think about the questions which go with each section, which
might go into surveys
Eric: Leave 1c and start with 1d
Shadi: The questions would be what do we want to do and what do we need to move forward
Eric: Back to 1c - what can we clarify to make this more workable
Detlev: Term baseline is ambigous - is it the
same as lowest common denominator
... From the wording it is difficult to know what is required, and what should
be defined
Eric, baseline is a list of several things at the moment
Detlev: We could have positive - browser x,
assistive tech y
... Or a lower version with lots of tech which needs to be supported
Kathy: Under WCAG accessibility support there are a number of considerations which we could take and make into a diagram
<ericvelleman> http://www.w3.org/TR/WCAG20/#accessibility-supporteddef
Kathy: would could also have a list of things
people have to think about when choosing a list of tech - what should you be
testing on
... There are further implications with regard to different techs
<ericvelleman> http://www.w3.org/TR/UNDERSTANDING-WCAG20/conformance#uc-accessibility-support-head
Kathy: we could put the information into a
diagram
... you cannot say test in this, this and this
Eric: we could copy from the document, but we have tried not to to date
<Detlev> aim for *simple* explanaton of options for defining baseline
Kathy: if you read the considerations documents there is a list of things
Eric: Like the idea of flow diagram
<shadi> Alistair: baseline is something you'd expect the website owner to define
<Detlev> XCan't undersatand Alistair!
<shadi> ...WCAG document is primarily designed for developers rather than evaluators
<shadi> ...really up to the Evaluation Commissioner in this situation
Detlev: Disagree
<shadi> [[This step ... is ideally carried out in consultation with the evaluation commissioner (who may or may not be the website owner) to ensure common expectations about the scope of the evaluation.]]
Detlev: We evaluate against BIVT, which fixes what we look at
<shadi> Alistair: but essentially up to the website owner what they define as their baseline
<shadi> ...putting a definition constrains them
Detlev: It is what you are looking at - but to simply take what the website owner states as sufficient is missing the mark
Eric: the question is who decides the baseline -
the commissioner or the evaluator
... it would be useful to discuss this further
Alistair: the commissioner sets the context of where they think their website will be used
Eric: We should leave this for further discussion
Kathy: as far as support, regardless of who
decides the baseline we still have a number of questions which would need to be
asked
... many people don't know, and many will need extra help from experts
... The baseline will differ from person to person - depending on who is using
the documentation
... We probably understand better that the commissioner as to accessibility
support
... It is going to be different in different contexts - the goal is to ensure
that it will work for the end user, in their different regions
Shadi: Agree with everyone - minimal requirements
are set by WCAG
... bar is set, below that you are not meeting WCAG
... This is a grey zone, where you can technically meet WCAG but in reality
this does not work broadly
... How to meet WCAG are decisions for the web developers, we can of course
provide a set of warnings
... We agreed from the beginning that this would not be adding additional
requirements
... We cannot be more prescriptive that WCAG, and we would hope to work with
the website owner to determine what baseline they were considering
... Warnings as opposed to what is technically required by WCAG
... WCAG sets the requirements
Eric: Look again at WCAG and see if we can clarify things a little.
<shadi> [[+1 to providing clarity *from the perspective of an evaluator*]]
<shadi> Alistair: issue is that setting the baseline by the evaluator might lead to differences
<shadi> ...can't get to an agreed baseline
<shadi> ...involvement of the website owner is necessary
Eric: 1c clarify steps or questions, and see if
we can add ownership
... can we atleast give people the questions - think about this when setting
baseline
Detlev: Evaluator should not set the baseline, it
should be more set by a group - disabilities groups- etc...
... The audience should create a consensus
... Some people like the latest things, but from a user perspective this might
not be the best thing
Eric: I'll look at the section and see what could be done - maybe a number of questions.
Shadi: Another issue or two: consistency
questions - does the baseline need to be consistent in a page, over a
website
... and that the baseline should be over all widgets, rather than one for one
and another for another
... We need to differentiate between WCAG failures and other issues
Eric: Summary, I will propose a list of
questions
... Surveys will be started for a number of comments
... Then everyone can have a look at the surveys
... next time 1d
... and scoring
Detlev: EIII project - evaluating websites - is
it possible to get input
... information on scoring would be great
Eric: We haven't talked about scoring yet
<shadi> http://www.eiao.net/publications/
Eric: The same discussions are being held everywhere, highlighting the difficulties with scoring
Tim: We're working on something
Shadi: Scoring is a very open ended discussion
... No rubber stamped solutions
... Other three things are easier to move forward with
Eric: Next telco next week
Eric: Survey will come in advance