Energy Systems

Electricity cannot be economically stored at utility scales. Supply and demand must be balanced at all times, and the structure of the transmission grid massive amounts of inefficiency. In order to deal with these issues, and to ensure reliable access to power, the global standard is to centralize control under Independent System Operators, who coordinate the grid for electric utilities. The efficiency of the electricity grid is also of fundamental importance for achieving lower carbon emissions, and reducing the impact of coal pollution on human health. Understanding all aspects of the electrical grid and their properties and complexities is a core focus for both Invenia and Invenia Labs.

Machine Learning

Invenia Lab’s machine learning is focused on systems with a fundamental importance to everyday life, but which are not dealt with efficiently. Our central interest is in optimizing complex decision-making and resource usage under uncertainty. In particular, the electricity grid offers data with unique properties, including many time series with complex structures, and operations that change rapidly. While our focus is on practical problems, we are also actively working on more fundamental research with a focus on AI, time series forecasting and decision-making. We are working to grow our academic presence at major conferences and on the pages of top journals.

Complex Systems

Complex Systems is a broad area of research focused on studying systems made of a large number of interacting components and the emergence of complex collective behaviors. Social systems, the brain, and electricity grids are examples. A related area of interest is Complex Adaptive Systems, referring to systems reorganizing themselves to solve complex problems. Neural Networks and Memristors are two such examples. We are focused on the practical aspects in the application of these tools, but also advancing the fields from a theoretical perspective.

Finance

Our interest in finance tools and techniques stems from our work in electricity grids, which provide unique challenges with transmission due to the physical properties of electricity.
On the practical side, we are studying risk and efficiency. An example of this work is our recent optimal leverage trajectories in the presence of impact in a marketplace. On the theoretical side, understanding the processes that improve the efficiency of the grid and the consequences of regulatory changes are also priorities. There are also many interesting connections between finance, statistical mechanics and complex systems to explore.

We are all researchers, and we come from many backgrounds including machine learning, mathematics, physics, computer science, engineering and economics. In addition to being researchers, we are all interested in problems that have a huge impact on the world, specifically complex systems that involve economic and environmental efficiency along with human health and AI safety.

Chief Technology Officer / Co-Founder

Christian Steinruecken

Christian completed his PhD under the supervision of Prof Sir David MacKay at the University of Cambridge (UK), and is a specialist in machine learning. He has led engineering projects in artificial intelligence, data compression and probabilistic programming. Christian believes that building intelligent technology is our best hope for making the world a better place.

Managing Director Invenia Labs / Co-Founder

Cozmin Ududec

Cozmin is a co-founder of Invenia, and is currently Managing Director of Invenia Labs in Cambridge. He received his PhD in the foundations of quantum theory from the University of Waterloo, and is still puzzling over the quantum world in his spare time.

Scientific Advisor / Co-founder

David Duvenaud

David is a co-founder of Invenia, and assistant professor in computer science and statistics at the University of Toronto. He received his Ph.D. in machine learning from Cambridge University. He has also worked at Google Research, the Max Planck Institute for Intelligent Systems, and the Harvard Intelligent Probabilistic Systems group.

Head of Infrastructure

Mike Himbeault

Mike has a degree in discrete and combinatorial mathematics and a computer and electrical engineering masters degree focused on computer network security. His Master’s degree involved developing a novel approach to detecting covert DNS tunnels that represented an improvement over existing methods. At Invenia, he manages the maintenance and evolution of the computing infrastructure supporting Invenia's operations and research.

Scientific Developer

Aron Hofer

Aron finished his Computer Science degree at the University of Manitoba specializing in Machine Learning. Aron has been with Invenia since 2009, where he is responsible for the efficient application of the organization's Machine Learning algorithms.

Research Advisor

Francesco Caravelli

Francesco's research focuses on statistical physics and complex systems, in particular complex networks, memristive circuits, econophysics and agent-based modelling. He is a theoretical physicist, interested in quantum and classical systems and the application of techniques of statistical physics and complexity to other disciplines such as economics, engineering and finance. He has been a Senior Researcher at Invenia Labs in Cambridge and a researcher at the London Institute for Mathematical Sciences, before moving as an Oppenheimer Fellow to Los Alamos National Laboratory.

Senior Researcher

James Requeima

James completed his Masters degree in Mathematics at the University of McGill. At Invenia, he works on our machine learning and risk management programs. He is currently working on finishing his masters in Machine Learning, Speech and Language Technologies in the department of Engineering at the University of Cambridge.

Senior Researcher

Lorenzo Sindoni

Lorenzo obtained a PhD in Theoretical Physics from SISSA (Trieste, Italy) for the last 6 years has been a researcher at the Max Planck Institute for Gravitational Physics in Potsdam-Golm (Germany), working on statistical approaches to quantum gravity. His main interests are emergent phenomena in many body physics, gravitational physics and complex systems on random graphs.

Senior Data Scientist

Mike de Denus

While working on his Master's degree, Mike developed a robotics system for maintaining formation movement with varying numbers of robots without the use of a centralized controller. His teams have won awards at numerous international robotics competitions. At Invenia, he focuses on the analysis and exploration of nodal and spot electricity markets.

Scientific Developer

Nick Thiessen

After completing BSc in computer science, Nick came to Invenia to work on building and maintaining machine learning systems and simulations. During his spare time, he can be found either developing, playing, or discussing games of all sorts.

Senior Developer

Rory Finnegan

Rory Finnegan joined Invenia as a Computer Science Co-op and Linux enthusiast with a background in Bioinformatics and Human Computer Interactions. Rory is currently completing a graduate degree in Computational Neuroscience.

Researcher

Wessel Bruinsma

Wessel completed his M.Phil. in Machine Learning, Speech, and Language Technology at the University of Cambridge. At Invenia, he conducts research in the field of machine learning and investigates applications thereof. Research interests include probabilistic modelling, Bayesian nonparametrics, approximate inference, and signal processing.

Researcher

Will Tebbutt

Will completed his M.Phil. in Machine Learning, Speech, and Language Technology at the University of Cambridge, where he focused on developing fast approximate inference methods for Gaussian Processes. When not working he can be found playing the guitar or listening to people play it well.

Advisor

Doyne Farmer

Doyne works with Invenia as a research advisor. Professor in the Mathematical Institute at the University of Oxford, and an External Professor at the Santa Fe Institute. His current research is in economics, including agent-based modeling, financial instability and technological progress. He was a founder of Prediction Company, a quantitative automated trading firm that was sold to the United Bank of Switzerland in 2006. His past research includes complex systems, dynamical systems theory, time series analysis and theoretical biology.

Technical Advisor

Zoubin Ghahramani

Zoubin works with Invenia as a research advisor. He is also a professor of Information Engineering at the University of Cambridge, where he leads the Machine Learning Group consisting of about 30 researchers, and the Cambridge Liaison Director of the Alan Turing Institute, the UK's national institute for Data Science. His academic career includes concurrent appointments as one of the founding members of the Gatsby Computational Neuroscience Unit in London, and as a faculty member of CMU's Machine Learning Department for over 10 years. His current research interests include statistical machine learning, Bayesian nonparametrics, scalable inference, probabilistic programming, and building an automatic statistician. He has published over 250 papers, receiving over 30,000 citations (an h-index of 74).

Explore the work we do through the papers published by people on the Invenia Labs team.

All

https://www.invenia.ca/wp-content/uploads/Correlations-and-Clustering-in-Wholesale-Electricity-Markets-thumbnail.jpg

Correlations and Clustering in Wholesale Electricity Markets

We study the structure of locational marginal prices in day-ahead and real-time wholesale electricity markets. In particular, we consider the case of two North American markets and show that the price correlations contain information on the locational structure of the grid. We study various clustering methods and introduce a type of correlation function based on event synchronization for spiky time series, and another based on string correlations of location names provided by the markets. This allows us to reconstruct aspects of the locational structure of the grid.

https://www.invenia.ca/wp-content/uploads/Correlations-and-Clustering-in-Wholesale-Electricity-Markets.pdf

https://www.invenia.ca/wp-content/uploads/edges.png

The mise en scène of memristive networks: effective memory, dynamics and learning

We discuss the properties of the dynamics of purely memristive circuits. In particular, we show that the amount of memory in a memristive circuit is constrained by the conservation laws of the circuit, and that the dynamics preserves the symmetry by means of a projection on this subspace. We obtain these results both for current and voltage controlled linear memristors. Moreover, we discuss the symmetries of the dynamics which are due to the circuit cohomology, and study the weak and strong non-linear regimes.

https://arxiv.org/abs/1611.02104

https://www.invenia.ca/wp-content/uploads/memristors.png

The complex dynamics of memristive circuits: analytical results and universal slow relaxation

Networks with memristive elements (resistors with memory) are being explored for a variety of applications ranging from unconventional computing to models of the brain. However, analytical results that highlight the role of the graph connectivity on the memory dynamics are still a few, thus limiting our understanding of these important dynamical systems. In this paper, we derive an exact matrix equation of motion that takes into account all the network constraints of a purely memristive circuit, and we employ it to derive analytical results regarding its relaxation properties. We are able to describe the memory evolution in terms of orthogonal projection operators onto the subspace of fundamental loop space of the underlying circuit. This orthogonal projection explicitly reveals the coupling between the spatial and temporal sectors of the memristive circuits and compactly describes the circuit topology. For the case of disordered graphs, we are able to explain the emergence of a power law relaxation as a superposition of exponential relaxation times with a broad range of scales using random matrices. This power law is also {\it universal}, namely independent of the topology of the underlying graph but dependent only on the density of loops. In the case of circuits subject to alternating voltage instead, we are able to obtain an approximate solution of the dynamics, which is tested against a specific network topology. These result suggest a much richer dynamics of memristive networks than previously considered.

https://arxiv.org/abs/1608.08651

https://www.invenia.ca/wp-content/uploads/2016/08/stat.png

Emergence of strongly connected giant components in continuum disk-spin percolation

We propose a continuum model of percolation in two dimensions for overlapping disks with spin. In this model the existence of bonds is determined by the distance between the centers of the disks, and by the scalar product of the (randomly) directed spin with the direction of the vector connecting the centers of neighboring disks. The direction of a single spin is controlled by a "temperature", representing the amount of polarization of the spins in the direction of an external field. Our model is inspired by biological neuronal networks and aims to characterize their topological properties when axonal guidance plays a major role. We numerically study the phase diagram of the model observing the emergence of a giant strongly connected component, representing the portion of neurons that are causally connected. We provide strong evidence that the critical exponents depend on the temperature.

http://arxiv.org/abs/1511.06512

https://www.invenia.ca/wp-content/uploads/2016/08/memr.png

Trajectories entropy in dynamical graphs with memory

In this paper we investigate the application of non-local graph entropy to evolving and dynamical graphs. The measure is based upon the notion of Markov diffusion on a graph, and relies on the entropy applied to trajectories originating at a specific node. In particular, we study the model of reinforcement-decay graph dynamics, which leads to scale free graphs. We find that the node entropy characterizes the structure of the network in the two parameter phase-space describing the dynamical evolution of the weighted graph. We then apply an adapted version of the entropy measure to purely memristive circuits. We provide evidence that meanwhile in the case of DC voltage the entropy based on the forward probability is enough to characterize the graph properties, in the case of AC voltage generators one needs to consider both forward and backward based transition probabilities. We provide also evidence that the entropy highlights the self-organizing properties of memristive circuits, which re-organizes itself to satisfy the symmetries of the underlying graph.

http://arxiv.org/abs/1511.07135

https://www.invenia.ca/wp-content/uploads/2016/08/tutte.png

On an ordering dependent generalization of Tutte polynomial

A generalization of Tutte polynomial involved in the evaluation of the moments of the integrated geometric Brownian in the Ito formalism is discussed. The new combinatorial invariant depends on the order in which the sequence of contraction-deletions have been performed on the graph. Thus, this work provides a motivation for studying an order-dependent Tutte polynomial in the context of stochastic differential equations. We show that in the limit of the control parameters encoding the ordering going to zero, the multivariate Tutte-Fortuin-Kasteleyn polynomial is recovered.

http://arxiv.org/abs/1512.02278

https://www.invenia.ca/wp-content/uploads/2016/08/conformity-driven.png

Conformity Driven Agents Support Ordered Phases in the Spatial Public Goods Game

We investigate the spatial Public Goods Game in the presence of conformity-driven agents on a bi-dimensional lattice with periodic boundary conditions. The present setting usually considers fitness-driven agents, i.e., agents that tend to imitate the strategy of their fittest neighbors. Here, fitness is a general property usually adopted to quantify the extent to which individuals are able to succeed, or at least to survive, in a competitive environment. However, when social systems are considered, the evolution of a population might be affected also by social behaviors as conformity, stubbornness, altruism, and selfishness. Although the term evolution can assume different meanings depending on the considered domain, here it corresponds to the set of processes that lead a system towards an equilibrium or a steady-state. In doing so, we use two types of strategy update rules: fitness-driven and conformity-driven. We map fitness to the agents' payoff so that richer agents are those most imitated by fitness-driven agents, while conformity-driven agents tend to imitate the strategy assumed by the majority of their neighbors. Numerical simulations aim to identify critical phenomena, on varying the amount of the relative density of conformity-driven agents in the population, and to study the nature of related equilibria. Remarkably, we find that conformity fosters ordered phases and may also lead to bistable behaviors.

http://arxiv.org/abs/1602.01808

https://www.invenia.ca/wp-content/uploads/Optimal-growth-paper-logo.png

Optimal growth trajectories with finite carrying capacity

We investigate the spatial Public Goods Game in the presence of conformity-driven agents on a bi-dimensional lattice with periodic boundary conditions. The present setting usually considers fitness-driven agents, i.e., agents that tend to imitate the strategy of their fittest neighbors. Here, fitness is a general property usually adopted to quantify the extent to which individuals are able to succeed, or at least to survive, in a competitive environment. However, when social systems are considered, the evolution of a population might be affected also by social behaviors as conformity, stubbornness, altruism, and selfishness. Although the term evolution can assume different meanings depending on the considered domain, here it corresponds to the set of processes that lead a system towards an equilibrium or a steady-state. In doing so, we use two types of strategy update rules: fitness-driven and conformity-driven. We map fitness to the agents' payoff so that richer agents are those most imitated by fitness-driven agents, while conformity-driven agents tend to imitate the strategy assumed by the majority of their neighbors. Numerical simulations aim to identify critical phenomena, on varying the amount of the relative density of conformity-driven agents in the population, and to study the nature of related equilibria. Remarkably, we find that conformity fosters ordered phases and may also lead to bistable behaviors.

http://arxiv.org/abs/1602.01808

https://www.invenia.ca/wp-content/uploads/2016/06/Neurogensis-Paradoxically.png

Neurogenesis Paradoxically Decreases Both Pattern Separation and Memory Interference

The hippocampus has been the focus of memory research for decades. While the functional role of this structure is not fully understood, it is widely recognized as being vital for rapid yet accurate encoding and retrieval of associative memories. Since the discovery of adult hippocampal neurogenesis in the dentate gyrus by Altman and Das in the 1960's, many theories and models have been put forward to explain the functional role it plays in learning and memory. These models postulate different ways in which new neurons are introduced into the dentate gyrus and their functional importance for learning and memory. Few if any previous models have incorporated the unique properties of young adult-born dentate granule cells and the developmental trajectory. In this paper, we propose a novel computational model of the dentate gyrus that incorporates the developmental trajectory of the adult-born dentate granule cells, including changes in synaptic plasticity, connectivity, excitability and lateral inhibition, using a modified version of the Restricted Boltzmann machine. Our results show superior performance on memory reconstruction tasks for both recent and distally learned items, when the unique characteristics of young dentate granule cells are taken into account. Even though the hyperexcitability of the young neurons generates more overlapping neural codes, reducing pattern separation, the unique properties of the young neurons nonetheless contribute to reducing retroactive and proactive interference, at both short and long time scales. The sparse connectivity is particularly important for generating distinct memory traces for highly overlapping patterns that are learned within the same context.

http://journal.frontiersin.org/article/10.3389/fnsys.2015.00136/full

https://www.invenia.ca/wp-content/uploads/2016/06/Correlation-Structure.png

Correlation Structure of Spiky Financial Data: The Case of Congestion in Day-Ahead Energy Markets

I study the correlation structure and argue that these should be ltered. I propose the use of dierent correlation measures other than Pearson, in particular a modication of Event Synchronization adapted to negative values or a ltered correlation matrix.

https://www.invenia.ca/wp-content/uploads/Internal2_Caravelli.pdf

https://www.invenia.ca/wp-content/uploads/2016/06/integrated-expontential.png

On Moments of the Integrated Exponential Brownian Motion

We present new exact expressions for a class of moments for the geometric Brownian motion, in terms of determinants, obtained using a recurrence relation and combinatorial arguments for the case of a Ito’s Wiener process. We then apply the obtained exact formulas to computing averages of the solution of the logistic stochastic differential equation via a series expansion, and compare the results to the solution obtained via Monte Carlo.

http://arxiv.org/abs/1509.05980

https://www.invenia.ca/wp-content/uploads/2016/06/Bounds-on-Transient.png

Bounds on Transient Instability For Complex Ecosystems

Stability is a desirable property of complex ecosystems. If a community of interacting species is at a stable equilibrium point then it is able to withstand small perturbations without any adverse effect. In ecology, the Jacobian matrix evalufated at an equilibrium point is known as the community matrix, which represents the population dynamics of interacting species. The system’s asymptotic short- and long-term behaviour can be determined from eigenvalues derived from the community matrix. Here we use results from the theory of pseudospectra to describe intermediate, transient dynamics. We show that the transition from stable to unstable dynamics includes a region of transient instability, where the effect of a small perturbation is amplified before ultimately decaying. The shift from stability to transient instability depends on the magnitude of a perturbation, and we show how to determine lower and upper bounds to the maximum amplitude of perturbations. Of five different types of community matrix, we find that amplification is least severe with predatorprey interactions. This analysis is relevant to other systems whose dynamics can be expressed in terms of the Jacobian matrix. Through understanding transient instability, we can learn under what conditions multiple perturbations—multiple external shocks—will irrecoverably break stability.

http://arxiv.org/pdf/1506.06971.pdf

https://www.invenia.ca/wp-content/uploads/2016/06/Geometric-Phases.png

Geometric Phases and Cyclic Isotropic Cosmologies

In the present paper we study the evolution of the modes of a scalar field in a cyclic cosmology. In order to keep the discussion clear, we study the features of a scalar field in a toy model, a Friedman-Robertson-Walker universe with a periodic scale factor, in which the universe expands, contracts and bounces infinite times, in the approximation in which the dynamic features of this universe are driven by some external factor, without the backreaction of the scalar field under study. In particular, we show that particle production exhibits features of the cyclic cosmology. Also, by studying the Berry phase of the scalar field, we show that contrarily to what is commonly believed, the scalar field carries information from one bounce to another in the form of a global phase which occurs to be generically non-zero.

http://arxiv.org/pdf/1411.7553v4.pdf

https://www.invenia.ca/wp-content/uploads/2016/06/multi1.png

Multi-scaling of wholesale electricity prices

We empirically analyze the most volatile component of the electricity price time series from two North-American wholesale electricity markets. We show that these time series exhibit fluctuations which are not described by a Brownian Motion, as they show multi-scaling, high Hurst exponents and sharp price movements. We use the generalized Hurst exponent (GHE, H(q)) to show that although these time-series have strong cyclical components, the fluctuations exhibit persistent behaviour, i.e., H(q)>0.5. We investigate the effectiveness of the GHE as a predictive tool in a simple linear forecasting model, and study the forecast error as a function of H(q), with q=1 and q=2. Our results suggest that the GHE can be used as prediction tool for these time series when the Hurst exponent is dynamically evaluated on rolling time windows of size ≈50−100 hours. These results are also compared to the case in which the cyclical components have been subtracted from the time series, showing the importance of cyclicality in the prediction power of the Hurst exponent.

http://arxiv.org/abs/1507.06219

https://www.invenia.ca/wp-content/uploads/2016/05/gcomp1.png

Ranking nodes according to their path-complexity

Thermalization is one of the most important phenomena in statistical physics. Often, the transition probabilities between different states in the phase space is or can be approximated by constants. In this case, the system can be described by Markovian transition kernels, and when the phase space is discrete, by Markov chains. In this paper, we introduce a macroscopic entropy on the states of paths of length k and, studying the recursion relation, obtain a fixed point entropy. This analysis leads to a centrality approach to Markov chains entropy.

http://arxiv.org/pdf/1410.2638v2.pdf

https://www.invenia.ca/wp-content/uploads/2016/05/memr1.png

Scale-free networks as an epiphenomenon of memory

Many realistic networks are scale-free, with small characteristic path lengths, high clustering, and power law in their degree distribution. They can be obtained by dynamical networks in which a preferential attachment process takes place. However, this mechanism is nonlocal, in the sense that it requires knowledge of the whole graph in order for the graph to be updated. Instead, if preferential attachment and realistic networks occur in physical systems, these features need to emerge from a local model. In this paper, we propose a local model and show that a possible ingredient (which is often underrated) for obtaining scale-free networks with local rules is memory. Such a model can be realised in solid-state circuits, using non-linear passive elements with memory such as memristors, and thus can be tested experimentally.

http://arxiv.org/pdf/1312.2289.pdf

We are interested in meeting the best and brightest to help us solve our most challenging problems. If you are interested in joining our team or want to collaborate on a research project, send us an email at team@invenialabs.co.uk.

Working at Invenia Labs

We are all researchers, and we come from many backgrounds including machine learning, mathematics, physics, computer science, engineering and economics.

Invenia Labs presents an opportunity to work with and learn from well regarded and respected individuals in machine learning, theoretical physics and mathematics, while contributing to research that has a direct impact on our society and the environment. Invenia’s team is composed of many highly skilled and brilliant individuals, who love what they do, and do what they love. In the simplest terms, we apply machine learning to solve the world’s most complex problems. We find great purpose in our ability to change the world for the better, reducing CO_{2} emissions and saving lives. It’s what drives us to work hard every day.

Current Opportunities

**Research Scientists**

Skilled individuals capable of conceptual, empirical and quantitative research. Strong mathematical skills, modelling skills, coding skills, and deep understanding of probability theory are essential. A typical research scientist candidate should have a Master’s degree or PhD in mathematics, computer science or physical science.

**Senior Research Scientists**

An exceptional and experienced individual capable of fundamental work in research, development and/or engineering, with a strong and proven track-record in academic communication, team-building and leadership skills. We look for expertise in one or more of: probability theory, statistics, machine learning, data science, algorithmic and/or probabilistic modelling, and algorithmic/probabilistic optimisation. Strong skills in mathematics and independent and collaborative coding are minimum requirements.

A typical senior research scientist candidate should have a PhD-level qualification in machine learning, computer science, statistics, mathematics, physics or a related field.

**Scientific Developers**

We are seeking a talented individual with excellent coding and software engineering skills, capable of understanding, extending, and building intelligently designed software projects. Communication and social skills are important. Strong skills in mathematics, probability theory, programming languages and computer systems are highly valued. A typical scientific developer candidate should have a Bachelor’s or Master’s degree in computer science, engineering or physical sciences.

How to Apply

If you are interested in joining our team, please send us your resume, cover letter, university transcript (for recent grads) and any other relevant information to team@invenialabs.co.uk.

Research Collaborations

We have some very interesting data sets and some very hard problems to solve. If you are interested in what we are doing and might want to collaborate on a research project with us, send us an email at team@invenialabs.co.uk.

LOCATE US ON THE MAP

Invenia

This is us, drop by sometime.