Test it on a whole specification book, and name only the equipment.
Do not point the software at your section. Hand any vendor, including us, the whole book, name the equipment, and see what comes back.
- Does it find every section that carries a requirement for the equipment? On the public Hailey book there were fourteen.
- Does every requirement come back with its page?
- Does it bring the motor, the starter and the control panel in from the electrical section?
- Does it surface the sentence that decides whether an equal may bid, like the one on page 2 of the Odessa specification?
- When two pages disagree, does it show both? In the Dixie Bend tank book, page 143 says no subsurface report exists and page 234 points to it.
Score it the way you would score a new hire. Did it find the item? Did it cite the page? Did it show you the conflict, or quietly pick a side?