Not wanting to steal the thunder of Deepmind, but I feel should mention my site which is currently beta as in "works for a few electricity power load and supply forecasting scenarios" and quite well so. Also, it's out there, cheap and fast.
https://ausblick.cryptoport.net
Unfortunately, it's only in German but the basic rule is:
- upload an excel (or csv) file
- wait a minute or so (for a typical dataset of say 3 years of hourly data)
- get back the results
There are a few design rules how the excel file should look like, e.g.,
- a date/time series per sheet (it accepts most English and European formats) and as many number series as you want (always use a decimal point instead of comma here)
- one series has to have consecutive missing numbers at the end, this will be the target value
Other than that, it should just work.
Feel free to comment.
PS: The examples might be the best explanation - https://ausblick.cryptoport.net/examples
- a number of smart features (usually a few k) depending on the series (using lags, aggregates, curve fits, combinations of features, ...)
- an iterative algorithm that selects features using maximum relevance (~ correlation with the target) / minimum redundancy and adds them to the model
- simple pca and ridge regression (because it's fast)
- a few optimizations of the final model (removing features, selecting a better ridge regression alpha with CV, ...)
The stack is pure Clojure / Clojurescript.
Interesting. Something I've long wondered about:
- Many wind farms are located in very specific places that have measured high winds and are otherwise good places to locate a wind farm.
- Global climate change is changing local climates too. Are the studies that were done 10-20 years ago to discover ideal wind farm locations no longer as accurate as they used to be?
- How are wind farm locations decided today? Is there is skill in predicting where a wind farm might be optimally located in 5 years rather than where it should be located today? Or if not (since probably not) what are the right risk mitigation actions to take if you are planning a wind farm but unsure about location?
- Since Machine Learning is useful in improving the value of the existing wind power, could it also be useful in this endeavor, finding optimal wind power generation locations?
- Lastly, is this something a person could research at home with open data and code? Just curious. :)
Actually, scratch that. On the scale of government, they're more medium size.
I don't mean to downplay these results. I’m not an expert. Just mentioning it in case someone at deepmind sees this and wants to reach out to PJM. I believe those models were created almost 10 years ago.
http://worrydream.com/ClimateChange/
The investment is not only in the machinery, but in tools to model, observe, and predict efficiency.
More info: https://medium.com/@astanway/introducing-amperon-2cded368284...
Is remote (Texas) viable?
Weather forecasting is based on physics-based models that, so far as I know, are essentially correct with the fundamental barrier to prediction being a combination of random effects and the difficulty of solving the complex equations numerically.
Deep learning involves training a heuristically approximate a system based on past data. It has been used to emulate various human-learning behaviors like recognizing images or recognizing good or bad position in games. Essentially, the visible successes are in "we don't know but can extrapolate roughly from data" (or we "know" but can't easily program it, in the case of image recognition). So I'd be surprised if deep learning do a good or better job in situation where we do know how things work, we already running algorithm which is correct - the situation with weather.
Unfortunately this is really only about predicting wind speeds to then forecast power output. IIRC, grid codes in some countries are starting to require 24 hour, 15 minute interval forecasts for renewables? I've also seen this feature being advertised by other companies over the years as well, although I don't recall seeing any numbers on accuracy for any of the models.
Would be nice to know a bit more detail? Which power grids was this approach tested on? How does this approach work exactly? Are we talking about DNN on time series? The graphs also really need labels to inform.
... is probably the operative part. Possibly uncharitable, but this appears to be ‘thought leadership’ marketing generally supporting the idea of GCP managed services.
Meteorologists have long used a metric, forecast skill, to evaluate different techniques. It would be good if the researchers used standard benchmarks.
ML is probably much more useful in understanding the demand for power, the reaction to weather and current events. But even still, it seems a stretch.
