Strength through pessimism! Keeping your stuff safe

Lock up your datum


"We have a lot of optimistic engineers - but not enough pessimistic engineers," reckons David Rosenthal. In the 1980s, Rosenthal designed the NeWS windowing system with James Gosling. In the 1990s he was NVidia's fourth employee, or really the first person the three co-founders hired. But for the past few years Rosenthal has been tackling an issue that's very close to Register readers' hearts - we know from your mail.

On Monday we discussed the permanence of digital media - or more accurately, the lack of it in our story, Digital memories: we can forget them for you wholesale!. Why should any of us trust our family albums to digital media such as a web photo service when we can't guarantee it'll be around?

In this age of wiki-fiddlers and other careless coders, where the line between marketing hypester and developer has been blurred, the premium that's placed on data integrity seems to be at an all time low. And the lossiest seem to work in "web services". Uh, look out!

This is where a glum approach can help, reckons David, who is not at all glum in real life.

"Pessimistic coders tend to be systems engineers who have been thinking about building fault tolerant systems for a long time, systems that are more resistant to human error," he told us on a UK visit this week. "It's a bigger problem than just the Web."

What Rosenthal has been building, with some success, is an open source peer to peer system that preserves digital content, even when powerful agencies want to see it altered or trashed. It's called LOCKSS - for 'Lots of Copies Keep Stuff Safe' - and it's already in use by 87 libraries. The project home is at Stanford University Library in northern Californian.

It's only addressing one of the problems of digital media - others include obsolescence and proprietary file formats, for example, but it's one that few people look at. And it's a very important problem to fix.

Lock up your datum

Digital formats, particularly web with publishing, are much worse than paper when it comes to preserving content for current and future users, and computers make it much easier to rewrite history, Rosenthal says.

"In 1984 Winston Smith's job was updating the only copy of history," he reminds us.

"There's a whole community of librarians who were desperate for this tool, we discovered," he says.

It's a P2P system in which multiple copies are made and the member nodes audit each other, and repair any damage. LOCKSS is both a web cache and a web crawler; the cache never gets flushed and the crawler is permanently scavenging.

"Good quality backup is difficult and expensive to do - and archive quality backup is really expensive and difficult to do", he says.

Many smaller libraries find getting system administration skills difficult and expensive, so the team turned LOCKSS into a turnkey product that boots off a BSD CD. It's really a network appliance now, he says.

Initially Rosenthal tried to interest potential users in one of the experimental "giant internet file systems" such as NEC's Intermemory, now OpenStore.

"We tried to sell them the idea of this big file system where no one knows where the data is - but the librarians didn't buy this story!"

Anyone can use the source code, but the team wants the system to be used by many more medium sized libraries before embarking on a rewrite of the LOCKSS' P2P protocols. Did he envisage a time when community groups would be able to use the software?

Rosenthal said he would expect it to be several years before it filtered down to the DIY level.

For now, he's seen an enthusiastic adoption by the humanities community, and built on early grant sponsorship from the National Science Foundation. LOCKSS is targeted at a specific community of people who take data integrity very seriously indeed. But it also sets a standard to which we must hold these new, highly emergent web services - before they lose all our valuable stuff. Backups aren't self-organizing.

Find out more about LOCKSS excellent work here. ®

Related stories

Digital memories: we can forget them for you wholesale!
Google - the only archive we'll ever need?
Archive.org suffers Fahrenheit 911 memory loss
How key Microsoft legal emails 'autodestruct'
Our kids deserve better than a Google future
German national library busts copy protection


Other stories you might like

  • Carnival Cruises torpedoed by US states, agrees to pay $6m after waves of cyber attacks
    Now those are some phishing boats

    Carnival Cruise Lines will cough up more than $6 million to end two separate lawsuits filed by 46 states in the US after sensitive personal information on customers and employees was accessed in a string of cyber attacks.

    A couple of years ago, as the coronavirus pandemic was taking hold, the Miami-based biz revealed intruders had not only encrypted some of its data but also downloaded a trove of data – names and addresses, Social Security info, driver's license and passport numbers, and health and payment information for thousands of people in almost every American state.

    It all started to go wrong more than a year earlier, as the cruise line became aware of suspicious activity in May 2019. This apparently wasn't disclosed until March 2020.

    Continue reading
  • India extends deadline for compliance with infosec logging rules by 90 days
    Helpfully announced extension on deadline day

    India's Ministry of Electronics and Information Technology (MeitY) and the local Computer Emergency Response Team (CERT-In) have extended the deadline for compliance with the Cyber Security Directions introduced on April 28, which were due to take effect yesterday.

    The Directions require verbose logging of users' activities on VPNs and clouds, reporting of infosec incidents within six hours of detection - even for trivial things like unusual port scanning - exclusive use of Indian network time protocol servers, and many other burdensome requirements. The Directions were purported to improve the security of local organisations, and to give CERT-In information it could use to assess threats to India. Yet the Directions allowed incident reports to be sent by fax – good ol' fax – to CERT-In, which offered no evidence it operates or would build infrastructure capable of ingesting or analyzing the millions of incident reports it would be sent by compliant organizations.

    The Directions were roundly criticized by tech lobby groups that pointed out requirements such as compelling clouds to store logs of customers' activities was futile, since clouds don't log what goes on inside resources rented by their customers. VPN providers quit India and moved their servers offshore, citing the impossibility of storing user logs when their entire business model rests on not logging user activities. VPN operators going offshore means India's government is therefore less able to influence such outfits.

    Continue reading
  • Hangouts hangs up: Google chat app shuts this year
    How many messaging services does this web giant need? It's gotta be over 9,000

    Google is winding down its messaging app Hangouts before it officially shuts in November, the web giant announced on Monday.

    Users of the mobile app will see a pop-up asking them to move their conversations onto Google Chat, which is yet another one of its online services. It can be accessed via Gmail as well as its own standalone application. Next month, conversations in the web version of Hangouts will be ported over to Chat in Gmail. 

    Continue reading
  • 5G C-band rollout at US airports slowed over radio altimeter safety fears
    Well, they did say from July, now they really mean from July 2023

    America's aviation watchdog has said the rollout of 5G C-band coverage near US airports won't fully start until next year, delaying some travelers' access to better cellular broadband at crowded terminals.

    Acting FAA Administrator Billy Nolen said in a statement this month that its discussions with wireless carriers "have identified a path that will continue to enable aviation and 5G C-band wireless to safely co-exist."

    5G C-band operates between 3.7-3.98GHz, near the 4.2-4.4GHz band used by radio altimeters that are jolly useful for landing planes in limited visibility. There is or was a fear that these cellular signals, such as from cell towers close to airports, could bleed into the frequencies used by aircraft and cause radio altimeters to display an incorrect reading. C-band technology, which promises faster mobile broadband, was supposed to roll out nationwide on Verizon, AT&T and T-Mobile US's networks, but some deployments have been paused near airports due to these concerns. 

    Continue reading
  • IBM settles age discrimination case that sought top execs' emails
    Just days after being ordered to provide messages, Big Blue opts out of public trial

    Less than a week after IBM was ordered in an age discrimination lawsuit to produce internal emails in which its former CEO and former SVP of human resources discuss reducing the number of older workers, the IT giant chose to settle the case for an undisclosed sum rather than proceed to trial next month.

    The order, issued on June 9, in Schenfeld v. IBM, describes Exhibit 10, which "contains emails that discuss the effort taken by IBM to increase the number of 'millennial' employees."

    Plaintiff Eugene Schenfeld, who worked as an IBM research scientist when current CEO Arvind Krishna ran IBM's research group, sued IBM for age discrimination in November, 2018. His claim is one of many that followed a March 2018 report by ProPublica and Mother Jones about a concerted effort to de-age IBM and a 2020 finding by the US Equal Employment Opportunity Commission (EEOC) that IBM executives had directed managers to get rid of older workers to make room for younger ones.

    Continue reading
  • FTC urged to probe Apple, Google for enabling ‘intense system of surveillance’
    Ad tracking poses a privacy and security risk in post-Roe America, lawmakers warn

    Democrat lawmakers want the FTC to investigate Apple and Google's online ad trackers, which they say amount to unfair and deceptive business practices and pose a privacy and security risk to people using the tech giants' mobile devices.

    US Senators Ron Wyden (D-OR), Elizabeth Warren (D-MA), and Cory Booker (D-NJ) and House Representative Sara Jacobs (D-CA) requested on Friday that the watchdog launch a probe into Apple and Google, hours before the US Supreme Court overturned Roe v. Wade, clearing the way for individual states to ban access to abortions. 

    In the days leading up to the court's action, some of these same lawmakers had also introduced data privacy bills, including a proposal that would make it illegal for data brokers to sell sensitive location and health information of individuals' medical treatment.

    Continue reading
  • Behold this drone-dropping rifle with two-mile range
    Confuses rather than destroys unmanned aerials to better bring back intel, says Ukrainian designer

    What's said to be a Ukrainian-made long-range anti-drone rifle is one of the latest weapons to emerge from Russia's ongoing invasion of its neighbor.

    The Antidron KVS G-6 is manufactured by Kvertus Technology, in the western Ukraine region of Ivano-Frankivsk, whose capital of the same name has twice been subjected to Russian bombings during the war. Like other drone-dropping equipment, we're told it uses radio signals to interrupt control, remotely disabling them, and it reportedly has an impressive 3.5 km (2.17 miles) range.

    "We are not damaging the drone. With communication lost, it just loses coordination and doesn't know where to go. The drone lands where it is jammed, or can be carried away by the wind because it's uncontrollable,"  Kvertus' director of technology Yaroslav Filimonov said. Because the downed drones are unharmed, they give Ukrainian soldiers recovering them a wealth of potential intelligence, he added.  

    Continue reading
  • TSMC may surpass Intel in quarterly revenue for first time
    Fab frenemies: x86 giant set to give Taiwanese chipmaker more money as it revitalizes foundry business

    In yet another sign of how fortunes have changed in the semiconductor industry, Taiwanese foundry giant TSMC is expected to surpass Intel in quarterly revenue for the first time.

    Wall Street analysts estimate TSMC will grow second-quarter revenue 43 percent quarter-over-quarter to $18.1 billion. Intel, on the other hand, is expected to see sales decline 2 percent sequentially to $17.98 billion in the same period, according to estimates collected by Yahoo Finance.

    The potential for TSMC to surpass Intel in quarterly revenue is indicative of how demand has grown for contract chip manufacturing, fueled by companies like Qualcomm, Nvidia, AMD, and Apple who design their own chips and outsource manufacturing to foundries like TSMC.

    Continue reading

Biting the hand that feeds IT © 1998–2022