EuroSciPy logo

EuroSciPy 2013

Brussels, Belgium - August 21-24 2013

Computing an Optimal Control Policy for an Energy Storage

Pierre Haessig

Sat 24 12:10 p.m.–12:20 p.m. in Dupreel


Computing an Optimal Control Policy for an Energy Storage

Pierre Haessig, Thibaut Kovaltchouk, Bernard Multon, Hamid Ben Ahmed, Stéphane Lascaud

Presentation of the talk

In this talk, we would like to present the methodology of numerically solving a problem of stochastic optimal control which is the Power Production Smoothing of an Ocean Wave Converters.

Please look at the attached draft paper for a better-shaped presentation of the scientific problem. We here focus more on how we would like to present the topic orally.

The background fields of this talk will be:

  • Electrical Engineering, in particular Renewable Power Generation (a field where Matlab is the most common tool...)
  • Dynamic Optimization, i.e. Discrete-time Optimal Control under Uncertainty, solved with the Stochastic Dynamic Programming method.

but the talk will put an emphasis on looking at the problem from a computational point of view (in particular for Dynamic Optimization), with also a bit on "how to do time series analysis with Python".

Power Smoothing of an Ocean Wave Converter will be the supporting example of the talk to demonstrate our main contribution: a small library to solve generic Dynamic Optimization problems.

Outline of the talk

Due to the nature of the Dynamic Programming method, a Markovian model of the system needs to be first created, before its optimal control policy can be computed. After the introduction, there will thus be two parts:

1) Stochastic Model of a Wave Energy production

First part will be thus devoted to the description and the fitting method of such a model based on an autoregressive (AR) model, with some "extras".

We will here make a quick tour of tools available in Python for time series analysis, which is a field where R is widely used.

We actually used both R, statsmodels.tsa and some custom-made fitting functions to implement a so-called "multi-lags autocovariance fit".

2) Optimal storage control with Dynamic Programming

Once the model is created, the Dynamic Programming equation will be numerically solved.

This is our main contribution: an attempt at creating a small library to apply the Stochastic Dynamic Programming (SDP) method to generic Dynamic Optimization problems. Indeed, it was quite often heard from experts of this method (which we are not) that for each new problem, one has to write its new SDP routine from scratch. This didn't sound satisfactory and we decided to try to write generic solver.

We will present shortly the API we have created, and how the equations are numerically solved. A key aspect of the genericity of our program is its ability to manage generic dimensions for both state and control variables. We will show how this was implemented using Cartesian products iterators from itertools as well as a good amount of vectorization & broadcasting.

A multivariate interpolation routine is a key building block of our program. Thus we will show a benchmark of existing interpolation tools (see which below). We are currently working on that part...

Finally, we will show the solution of the optimal control problem of our running example, the Power Smoothing of an Ocean Wave Converter. Optimal solution will be compared to a simple heuristic control policy (see fig. 1 of the paper).

Software tools & modules used

  • numpy quite exclusively for the all "number crunching" tasks:

    • discrete-time dynamical system simulation
    • fitting an autoregressive model with the so-called "multi-lags autocovariance fit"
    • numerical solving Bellman's Dynamic Programming equation (main contribution)
  • itertools to make the the program generic with regard to the system dimension

  • inspect, to bring a bit of introspection which makes the API funnier
  • matplotlib to plot and analyze time series.
  • statsmodels.tsa for time series analysis (and concurrently a bit of R...)
  • interpolation routines (N-D interpolation) from scipy.interpolate and others being under investigation such as Pablo Winant's work on a Cython multilinear interpolation routine (