2020-12-18 Discovery Functional Committee Meeting notes

1:00PM

Call to order/
welcome

Old business:

VE evaluation plans

• @Christine Holmes has collected all Go VE materials from Ex Libris on this page: https://calstate.atlassian.net/wiki/spaces/ULMSD/pages/1601601537

• @Gabriel Gardner (Unlicensed) has emailed the ULMS project managers list to establish contact people who are the PBO admins at each campus; this list will facilitate evaluation communication. Contact names have been added to the CSU Primo Instances page: https://calstate.atlassian.net/wiki/spaces/ULMSD/pages/78291823
  
  

VE analysis update: load time

Paul H.

• How “deep” do we want to test?

• GTmetrix https://gtmetrix.com/

  • Gave more detail

  • Paul says this is easier to use than others he looked at

• Testing ULRs

  • simple search

    • local scope

    • a scope that also has CDI results

  • a full record

  • Local data may come back faster - Christian notes

• Will need to test 3 separate times to make sure the results are representative and not affected by various internet anomalies

  • will need a field to note which test (1, 2, 3 they are entering data for)

  • testing should be done when we are in session, for realism purposes

  • testers should try to test at the same times

• Users should not be signed in

• Record data via a Google form
  
  

VE analysis update: back end usability

Christine H.

• (See spreadsheet from previous meeting with potential exercises.) https://calstate.atlassian.net/wiki/spaces/ULMSD/pages/1562443777/2020-11-20+Functional+Committee+Meeting+notes?focusedTaskId=5

• Have a variety of common tasks

  • Have users pick 5 tasks to perform and then rate in terms of (easier, same, harder) to do e.g.

    • adding/changing facets

    • adding/changing labels

    • view changes

  • At some point in the testing phase, we will check in on what people have done and make sure that each task has had some people test it.

• Some questions will need Likert scale responses so we can have some quantitative aspect

• Record data via a Google form

VE analysis update: front end usability

Cathy O.

• Cathy put together a form detailing tasks https://forms.gle/PWPw7p9irsJFaYfj9

• Need to look at local data affected by norm rules (e.g. book result in full record display) and an article for CDI data.

• Questions will focus on the display experience rather than search results

VE analysis update: results/relevancy

Andrew C.

• See Andrew’s plan linked from previous meeting.

  • Two phased plan: Phase 1 Does VE affect relevancy ranking at all? Phase 2 Test precision and recall. Possible 3rd phase with qualitative follow-up discussion between testers.

  • In 1st phase campuses will come up with their own queries and compare in same scopes across Primo / VE

    • Compare top 20 results

    • ID the scopes used for testing

    • collect results in an Excel file

    • note changes/differences between sets

  • Directions will need to be clear (for everyone not just this task from Andrew).

  • In possible 2nd phase we might look at precision (precision = number of relevant results/ total results)

    • Second phase only needed if results from Phase 1 indicate big differences.

Old business:
next Open Forum

@Gabriel Gardner (Unlicensed)

Next open forum will be Jan 22 or Jan 29 depending on the information learned at the Go VE kickoff meeting with the main testing campuses will be Monday, Jan. 11th, and the progress achieved by the Discovery Functional Committee meeting on Jan. 15th.

• @David Walker will figure out who needs to be invited to the Kick-Off meeting.

Old business: custom resource type icons

Tabled

New business:

ULMS Palooza Discussion

@Gabriel Gardner (Unlicensed)

Do we want a ULMS virtual palooza event this summer? Need to get committee's opinion and communicate it back to steering cmte.

• Might be good to have a refresher meeting over Zoom. Depending upon where we are in Primo VE testing, it might good to participate and inform people about this via the Palooza event.

New business:

Adding misc. metadata collections to consortial Primo

@Gabriel Gardner (Unlicensed) @David Walker

ULMS Steering committee has tasked Resource Management and ERM Functional Cmtes. with developing a policy on inclusion of OA/free metadata collections in Primo. (Whether these will be piped in directly or added to Alma is TBD.) Does Discovery Functional Cmte. have any role to play in this?

• Bad metadata results in bad UX.

• DFC will leave this in the capable hands of RMFC and ERMFC.

New business:
Content notes/'trigger' warnings

@Gabriel Gardner (Unlicensed)

• This is a developing area; tabled until further developments require our action.