A community benchmark for the residential DER lifecycle — now an LF Energy Sandbox Project
Smart HEMS Benchmark has been accepted as an LF Energy Sandbox Project. Initiated by EcoFlow and led by Dr. Rui (Ricky) Li, with co-initiators Dr.-Ing. Anurag Mohapatra (TUM CoSES) and Prof. Tao Sun (Shanghai Jiao Tong University), the project delivers the open, reproducible evaluation standard that residential distributed energy resources (DERs) have long lacked, and it now lives in a neutral, community-governed home built for exactly this purpose.
The challenge that brought us together
In recent years, we have witnessed unprecedented growth in distributed energy resources, particularly residential solar and battery systems. By 2030, solar PV is projected to be the largest single source of renewable capacity growth (IEA), and residential battery storage is expanding rapidly on the back of falling costs, higher electricity prices, and stronger incentives (McKinsey & Company).
While this is an exciting trend, it brings a pressing challenge: the absence of standardized scenarios and frameworks for evaluating the whole lifecycle, from the location and sizing of DERs, to Home Energy Management System (HEMS) optimization, to grid-support services at the home and community level. Today, these decisions are each measured in isolation, on proprietary datasets, using incompatible assumptions. Proprietary systems create silos that limit innovation, and the lack of transparency not only hinders healthy competition but also delays the development of truly effective solutions.
What the Smart HEMS Benchmark provides
The Smart HEMS Benchmark models DER decision-making as an optimization problem spanning three complementary phases and time horizons from a single day to roughly fifteen years:
|
Phase |
Decision |
Primary concern |
Time horizon |
|
Location & Sizing |
Installation location and capacity |
Electricity-cost reduction; backup power |
DER lifecycle (~15 years) |
|
HEMS (single household) |
Charging / discharging behavior |
Self-consumption; self-sufficiency; comfort |
Typically 1 day |
|
Grid Support |
Capacity reserved for VPP participation |
Additional revenue from grid markets |
Typically 1 year |
Across these phases, the project provides shared scenarios, open datasets, common metrics, and reference algorithms, including realistic PV capacity configurations, storage specifications, and daily load profiles that reflect real-world conditions. This creates a level playing field where HEMS strategies, sizing tools, and grid-support approaches can be evaluated transparently, reproducibly, and independently of any single manufacturer.
Why we built this in the open
The project traces back more than a decade. Dr. Rui (Ricky) Li, IEEE Senior Member, former Chief AI Expert at EcoFlow and the project’s initiator, has worked at the intersection of energy and AI since installing his first distributed solar-plus-storage system on a university campus in 2012, and has practiced open source in energy since his PhD.
|
“The AI community’s openness far outpaces that of the energy sector. I’ve committed to open source in energy since 2014, yet the sector long lacked a neutral home for shared, working code; LF Energy is exactly that home. It is a privilege to bring the Smart HEMS Benchmark here, and I believe transparent, reproducible benchmarking is how residential DERs earn trust at scale: a shared standard that lets researchers, vendors, and households compare their energy systems on equal footing.” |
We chose open source because transparent, reproducible benchmarking is the fastest route to trust at scale: open datasets, scenarios, metrics, and algorithms enable manufacturer-neutral comparison, reduce market fragmentation, and let the whole community co-develop scenarios, controllers, and validation methods.
Why LF Energy is the right home
EcoFlow has been an LF Energy General Member since 2025, and the Smart HEMS Benchmark is the first project we are contributing to the foundation. That choice reflects a shared philosophy: EcoFlow has always believed energy independence shouldn’t be a luxury — it should be a fundamental right — which aligns naturally with LF Energy’s mission to create an open, interoperable energy ecosystem.
Just as Linux revolutionized computing by fostering collaboration over competition, we see immense potential in applying that model to energy systems. LF Energy builds shared digital infrastructure for the energy sector through open collaboration, working code, and neutral governance; a trusted home where no single company controls the roadmap. The Smart HEMS Benchmark connects naturally to LF Energy’s work on DER flexibility, grid simulation, interoperability, and AI.
EcoFlow’s Director of AIoT, Edison Tuo, frames the contribution:
|
“At EcoFlow, we recognize standardized benchmarking as fundamental to scaling intelligent, low-carbon residential energy systems. By contributing our open source HEMS Benchmark project to LF Energy, we provide a consistent, reproducible suite to assess HEMS energy efficiency, scheduling optimization, renewable integration, and operational reliability — a transparent baseline for the global energy community. EcoFlow will maintain long-term contributions to drive continuous tooling improvement and scenario expansion.” |
A global, cross-institution effort
The benchmark brings together complementary expertise spanning the residential-DER lifecycle, with each partner leading the phase it knows best. Co-initiator Prof. Tao Sun — now Associate Professor at Shanghai Jiao Tong University, and a postdoctoral scholar at Stanford University when the project began — anchors the location and sizing phase through his systematic feasibility assessment of residential PV and storage, published in Nature Energy. EcoFlow leads HEMS algorithm design and optimization. And the Technical University of Munich CoSES, with its multi-family, multi-carrier (heat and electricity) Power-Hardware-in-the-Loop (PHIL) platform and annual HEMS Symposium, extends evaluation toward community and grid-support scenarios.
For co-initiator Dr.-Ing. Anurag Mohapatra, Group Leader at TUM CoSES, a peer-reviewed benchmark is the missing piece for adoption:
|
“HEMS are a necessary behind-the-meter intervention for our decarbonization goals, but a private customer makes this investment in return for a long-term reduction in their electricity bill, and that investment must come with a guarantee on potential annual savings to accelerate the adoption curve. We need the equivalent of the WLTP test for electric vehicles: a peer-reviewed benchmark that compares HEMS algorithms for a specific building type, irrespective of differences in energy systems, components, forecast accuracy, or geographical and seasonal variation.” |
Co-initiator Prof. Tao Sun sees the same need from the customer’s side:
|
“Distributed energy resources have become essential to end users’ energy affordability and resilience against disruption, but their benefits vary widely from household to household and region to region, so when different contractors and HEMS providers all claim broad effectiveness they are rarely comparing apples to apples, leaving customers unable to identify what truly fits their own needs. An open source, transparent, and fair HEMS benchmark is exactly what this field needs: by making DERs and HEMS from a wide range of providers directly comparable, it turns technology improvement into something quantifiable and gives customers, industry, and researchers a common, evidence-based basis for better decisions.” |
Looking ahead: milestones and commitments
Having introduced the initiative at the 2025 LF Energy Europe Summit — the near-term roadmap focuses on community-defined benchmark scenarios and an expanding pilot evaluation platform for residential energy management. Over the coming year the Smart HEMS Benchmark community aims to co-develop shared standards for DER integration in HEMS, growing the open library of scenarios, datasets, and reference controllers alongside vendors, researchers, utilities, and testbed operators.
The long-term vision extends toward 2030: a thriving open source ecosystem where algorithms and data are shared, tested, and improved collectively. This isn’t just about technical standards, it’s about fostering a culture of collaboration that can accelerate the global transition to sustainable energy. EcoFlow is committed to long-term, active maintenance of the project, driving continuous tooling improvement and scenario expansion in partnership with the community.
|
“Every HEMS vendor claims to be the best, but the industry has no common way to prove it. The HEMS Benchmark gives everyone a fair, reproducible, vendor-neutral yardstick, and LF Energy is the right home to keep it trustworthy.” — Dr. Xiaoke Yang, Chairperson, LF Energy Smart HEMS Benchmark; AI Technology Lead, EcoFlow |
|
“In an industry where every vendor claims victory without a shared yardstick, the HEMS Benchmark exists to replace marketing noise with reproducible science, so the best algorithm wins on merit, not on marketing budget.” — Xiaochun Zhang, Chief Architect, LF Energy Smart HEMS Benchmark; Head of AI Platform, EcoFlow |
A call to the community
The challenges of the energy transition are too complex and urgent for any single company to solve alone. We invite companies, researchers, developers, utilities, and energy enthusiasts to use the benchmark, challenge it, and extend it, so the best solutions win on evidence, and households everywhere can trust the systems powering their homes.
The future of energy is open, collaborative, and sustainable. At EcoFlow, we’re proud to build it here, together with Linux Foundation Energy and the wider community.
Join us in powering the future, together.