The city of Berkeley has officially launched a new website featuring 17 data sets related to everything from municipal water usage and employee salaries to crime heat maps, energy consumption, restaurant inspections, registered business licenses and much more.
The city unveiled the website, which began on a pilot basis in December, on Thursday. City spokesman Matthai Chakko said the most exciting thing about the project is what the public might do with the information now that it’s available.
“What can be done with open data is limited only by the imagination,” said Dee Ridley-Williams, the city’s new interim city manager, in a prepared statement. “We’re excited to see how the Berkeley community will utilize this new tool.”
The website features downloadable spreadsheets, charts, maps and more. The data can be embedded online, filtered and manipulated in different ways. The city offers a range of formats for those who want to download the information to allow flexibility in how it is used. There are also tutorials for those wanting guidance about how to approach what’s posted.
See the city’s new open data portal here.
Each dataset has a handy “about” section that features not only additional information about what the data includes, but also a direct way to contact the owner of the data to find answers to specific questions. There are many features on each page, related to social media sharing, different ways of viewing the data and different ways to manipulate it, but the interface is user friendly for those who want to explore it.
The city has created a special interface focused on its 311 calls for service — which run the gamut from calls about refuse and recycling to street conditions, graffiti and vandalism, account services and billing and more. According to the 311 data, there have been 90 Public Records Act requests submitted to the city in the past two months. It’s not possible to see what was requested, but categories for each request are provided.
Another special interface related to the city budget is still under construction but should eventually be available once the kinks are worked out. (Those interested to see what it might eventually look like can peruse a similar site created by the city of Boston.)
Donna LaSala, the city’s outgoing director of information technology, said the city’s commitment to transparency and open government drove the effort to create the portal.
She said, in other cities, the public has created apps showing the best routes to parks, including public transportation, and easy ways to see bike routes around town.
The city partnered with Socrata, a company that made a name for itself by launching the White House’s open data portal. The company has created 80%-90% of all the open data portals in the U.S. public sector, she said.
The city’s Department of Information Technology spearheaded the project, working with staff members from all city departments, as well as about 10 student volunteers from the Presidio Graduate School of Management — where Lasala was teaching, and where she is set to begin a new job this month — along with a couple students from the University of California, Berkeley’s School of Information.
“They spent hundreds of hours building, testing, and tuning the data portal based upon community feedback,” according to a statement released by the city Thursday about the launch.
Lasala said the students contributed more than 1,500 hours of volunteer time to the project. She said there had been almost no cost to the city beyond staff time, which she estimated to have been about one hour for every five hours contributed by the student interns. About a dozen staffers from all city departments helped oversee the interns and worked closely with them on the project.
“It was a labor of love for a lot of city staff that really believe in this,” Lasala said.
She said the biggest challenge that arose as a result of the effort was to get the city’s existing data into a format that could be fed into the new portal. That was particularly challenging due to the city’s “legacy systems,” which are used for different types of data collection and storage.
“Taking a 1980s system and making it integrate with a modern open data portal, it takes a lot of programming work. And getting those data extractions to update automatically: It’s a big effort,” she said.
Lasala said former City Manager Christine Daniel estimated in 2010 that it would cost about half a million dollars over numerous years to update the legacy systems and replace them with modern software. The city is working to replace those systems but the going is slow. As improvements are made, it should be easier to get the city’s data into formats that can be more easily shared with the public.
Lasala said Socrata, too, is likely to make further refinements and improvements as time goes by.
“I’m just really proud that we’ve gotten to the point where some of our legacy data can be presented in a way that’s useful,” Lasala said. “It’s exciting to see what people will do. We have a very smart and engaged community — and creative. They will come up with uses that we might not have thought of. We hope that’ll happen.”
According to the city, “By providing raw data used in municipal operations, the City of Berkeley hopes to increase government transparency, civic engagement and accountability. Depending on the innovations created by community members, Open Data can increase efficiency and innovation. Data sets on the site reflect the most frequent Public Records Act requests. By providing those data sets proactively, the City also aims to reduce staff time extracting and formatting data.”
According to the city, other local agencies that have created open data sites include San Francisco, Oakland and Alameda County. Boston, New York, the White House, the State of California Comptroller’s Office, the California State Board of Equalization and the California Department of Public Health have also launched open data sites.
Community members can sign up to receive email alerts when new data is added to the Berkeley site, by clicking the orange RSS feed button at the top of the list of data sets, and can also submit suggestions for data sets they would like to see included, as well as feedback about what’s already available.
Some of the data sets cover a particular time period — such as those for employee salaries for 2011-13 — while others, such as the police calls for service heat maps, will update automatically to include the most recent six months of data.
For those interested in the calls for service data from the Berkeley Police Department, the heat maps offer a new way to look at the frequency of those calls in a visually appealing manner. The city had previously posted spreadsheets listing those calls on a daily basis, but there was no way to look at the data as a whole, or to visualize or sort it automatically. (NB: Calls for service do not reflect confirmed crime reports, and represent “unverified, preliminary information,” the department explains.)
For more background about the city’s open data work, background on the open data movement and more, see this lengthy staff report to the Berkeley City Council and the Open Data Handbook. The open data portal can be accessed here.
Do you rely on Berkeleyside for local news? Support independent journalism by becoming a Berkeleyside member for $10 a month or even less, or by making a one-time donation.
