This article is more than 1 year old

Yahoo! Gits! Web! Security! Scanner!

Want to scan a million pages? Here's how

Yahoo! has opened the kimono on a project it hopes will make the Web that little bit safer for everyone, Project Gryffin.

The project's so new, its to-do list includes “Link to news/finance scan video” – and, El Reg would add, someone might go over the project's GitHub page with an editor's pencil in hand.

The company describes the project as a large scale Web security platform designed to solve two specific problems: coverage, and scale.

Scale is obvious: the Web is a big place. Coverage is more complex, the post notes: “Coverage has two dimensions - one during crawl and the other during fuzzing”.

During crawl, Gryffin's designed to see as much of a Web app's footrpint as possible, while during fuzzing, the challenge is to test “each part of the application for [an] applied set of vulnerabilities”.

The crawler is designed to discover the “millions of URLs” that might be generated by a single template from just on of the URLs. There's also a de-duplication engine, and PhantomJS is used to handle DOM rendering in client-side JavaScript apps.

A combination of open source and custom fuzzers are used in Gryffin, Yahoo! says, hinting that some of its own work might turn up in the code base in the future.

Gryffin's requirements are Go, PhantomJS v2, the NSQ distributed messaging system, Sqlmap and Arachni for fuzzing, and Kibana and Elastic Search. ®

More about


Send us news

Other stories you might like