[Noisebridge-discuss] PG&E Tools
Daniel at coloraura.com
Sun Mar 4 20:21:08 UTC 2012
Taking a csv and stuffing it into a data store is an easy problem to solve, which doesn't require a web scrapper at all. If you need help automating the fetching of the csv, chances are you don't need to scrape, just simulate the login process. curl is a useful unix tool for such a thing.
It is unfortunate that pg&e doesn't have a public restful API for access though. Perhaps a request to them is in order. I've been thinking about that for my own personal purposes.
Sent from my iPhone
On Mar 4, 2012, at 10:02 AM, Ben Kochie <superq at gmail.com> wrote:
> Yes, that's what I'm talking about. We need to grab the files there
> and import them into a datastore of some kind.
> On Sun, Mar 4, 2012 at 09:33, miloh <froggytoad at gmail.com> wrote:
>> On Sat, Mar 3, 2012 at 11:43 AM, Ben Kochie <superq at gmail.com> wrote:
>>> I've spent some time digging around for open source tools for
>>> accessing PG&E data. It looks like there is no API or anything useful
>>> for getting access to the data. We'll have to write a web scraper to
>>> pull the data. Since this isn't something I'm good at, I'm looking
>>> for some help. From what I can tell it shouldn't be terribly hard to
>>> grab the data we need and then feed it into a database.
>>> Noisebridge-discuss mailing list
>>> Noisebridge-discuss at lists.noisebridge.net
>> a web scraper?
>> for my consumer account I download the usage data in a .csv
>> you don't see it for the noisebridge account?
>> something like the following:
> Noisebridge-discuss mailing list
> Noisebridge-discuss at lists.noisebridge.net
More information about the Noisebridge-discuss