Revealing Obscured Government Data

I experienced a prime example of an obscured government on a research trip to Albany, New York, in the spring of 2009. I spent two whirlwind days in New York’s capital learning the ins and outs of the state’s legislative information management. Two days in Albany had me feeling like I’d gone back in time, to somewhere around 1972. Albany’s mismatched modern architecture juts out of the hillside on which the medieval-castle-like state capitol sits. Walking into the lobby room between the Assembly and Senate chambers, I was shocked to see that it’s actually full of lobbyists. Sharply dressed in suits and shined shoes, they anxiously awaited Assembly or Senate leaders to show up. They huddled in conversation on the large wood and green velvet couches. In a corridor nearby, offices of New York newspapers bustled with actual reporters who seemed straight out of central casting. If you’ve toured or worked in other legislatures, you’d be shocked by the scene in New York. The state is an anachronism—nobody’s lobbyists congregate in the lobby area, and the press barely exists inside legislative bodies anymore. Hallways of government are usually quiet and empty—everyone is working online from somewhere else. I’d discover that lobbyists congregated at the capitol to get an edge on valuable legislative information, a precious commodity in New York.

Sitting in the Assembly’s gallery, I looked out on a small sea of state Assembly members engaged in voting on bills. I opened the Assembly’s bill tracking website, pulling up publicly available information on each bill that came before the Assembly. What I found online was shocking; there was no mention of any of these bills coming up for vote that day—no schedule, no notice, nothing beyond the first draft of a bill and maybe one other copy from committee. What I’d find out on my tour of Albany’s legislative system is that the only people who knew bills were coming up were those in the Assembly, the partisan-appointee clerks[160] who control the release of legislative information, and the lobbyists who paid the state $2,500 to access the full record of New York legislation from the Legislative Research Service (LRS).[161] New York may be the most expensive state to gain access to accurate legislative information, but it’s joined by other states that view access to legislative information as a means to political power or increasing the legislature’s funding.

Over the next day, I found that the New York State LRS sells legislative information at a per-session subscription of $2,500, with the funding going to supporting offices within the bill drafting and publication process of the state. I found that transcriptions of minutes from Senate votes and debates were copied onto microfiche with 35-year-old machines and filed in drawers, and only one computer was available for looking up where the transcripts were in the file drawers. For copies of transcripts, a citizen would need to request them at twenty-five cents per page—perhaps not exorbitant fees, but certainly prohibitive of timely access to records. I went from one office to the next, learning that information about the state’s legislation was spread across several offices, each of which was bogged down in old technology, or stuck in red-tape rules with which the majority party could refuse to release information. New York may be at the extreme end for lack of transparency and accessibility, but the state is not an anomaly. Governments across the United States and the globe are struggling to modernize their technology and become more transparent. Luckily for the citizens of New York, the state Senate’s new CIO, Andrew Hoppin, and his crackerjack team of techies came on board in February of 2009 and began taking the Senate toward transparency and modern technology.

Recently, I caught up with Hoppin to find out what progress his office has made in the New York State Senate. The improvements his office has made in just 12 months are a lesson for all governments. The Senate’s quick shift toward transparency, information modernization, and open information policies have radically improved the information publication within the state, catapulting them from a 1970s framework with a dash of late-1990s website technology to a full-fledged modern legislative body.

Hoppin and his crew are part of the political-appointee structure of the New York State legislature; unlike many legislatures, the administrative and clerical staff work for the party in power. With the 2008 election of a new Democratic majority, the New York Democratic Senators made a commitment to transparency and modernization. Making good on their promises, they brought Hoppin in to modernize the Senate (it should be noted that the Assembly and the Senate are not guided by the same technology or transparency policies—they set their own agendas). Quickly, Hoppin assembled a fresh team of techies with a bent toward doing things inexpensively while utilizing open source tools and well-adopted data standards. The team researched how information flowed through the Senate and (something I often suggest to governments) found where the information was in its earliest data form.

An example of that early data is the Senate’s transcripts, typed up by Candy, of Candy Co., a one-woman transcription wonder with a locked-in contract for transcription services with the Senate. Her transcripts went through a process in Microsoft Word to be published only on the Senate intranet[162] and into the microfiche process I described earlier, resulting in microfiche rolls of transcripts that were neatly tucked into a file drawer and made accessible to those who made a pilgrimage to the office to pay for copies. To add those transcripts to the online records of the Senate, Hoppin’s office posted them at the gorgeous and comprehensive http://nysenate.gov/legislation.

Through Knowledge As Power’s research in 2009, I noticed a trend amongst legislatures of all sizes—either their electeds or the techies realized their technology wasn’t sufficient to meet the needs of a modernized constituency seeking greater access and transparency. Without partners in technology and policy, modernization stagnated and often resulted in governments remaining locked into old technology or spending a lot of money contracting out to inadequate third-party software systems.

The New York State Senate, in less than a year, with legislative leadership and a fresh technology team, overcame what I would call the most complicated, tension-filled, overly politicized legislative environment and anachronistic technology I’ve ever seen. They are quickly progressing toward more transparency, access, and modernization.

Now, the New York State Assembly—and other legislatures—have an example to follow.



[160] New York’s legislative clerical staff would typically be nonpartisan support staff in other state legislatures. However, in New York, these staff (like those who manage updates to legislation public records) are selected by the majority party, allowing for information management by a party’s staff.

[161] LRS is not affiliated with political parties, but rather is under the “Legislative Bill Drafting Commission” appointed by both houses of the Legislature. LRS’s sale of the legislative information provides revenue for general bill drafting services.

[162] Accessible only within the Senate’s shared database.