EMS Benchmark Awards 2026

Benchmark methodology

Energy management systems coordinate generation, storage and consumption. Their assessment should establish how well they perform that task, what their innovation adds and whether the results hold under everyday operating conditions.

Commercial and industrial installations and households have different requirements. A factory may need to respect a contracted import limit while maintaining production and responding to flexibility requests. A household may need an electric vehicle ready by morning, a comfortable indoor temperature and a lower electricity bill. Each award therefore needs its own test scenarios and weighting.

Assessment criteria

The assessment assigns 25% to demonstrated innovation and 20% to measured operating value in both categories. The remaining criteria reflect the conditions in which the system will be used.

Demonstrated innovation

Each entrant should identify a specific technical development, the problem it addresses and the existing practice against which it should be assessed. Examples include improved coordination of several assets, useful operation with limited connectivity, or a substantial reduction in the work needed to integrate new equipment. The feature must be available in the submitted version and its claimed improvement must be reproducible. Planned features receive no performance credit.

The innovation score considers technical distinctiveness, practical relevance and applicability beyond one demonstration. Financial gains are scored under operating value. The use of AI, the number of dashboard features and the size of the supplier are not scoring criteria.

Measured operating value

Compare the EMS with a documented reference controller on the same installation. Report the difference in net electricity cost, peak demand, energy flows and battery use. For C&I systems, include the effect on site operations and contracted grid capacity. For residential systems, maintain the same comfort settings, charging deadlines and reserve requirements.

Calculations should include applicable import and export tariffs, taxes and levies, network charges, transaction or aggregator fees, EMS subscriptions and an explicit estimate of battery wear. Apply the same cost boundary to the reference controller. Report one-off integration costs separately and include them once in a five-year cost comparison. Show annual net value and five-year total cost separately, with sensitivity to energy prices, fees and degradation assumptions. Research on REopt illustrates why degradation costs and perfect knowledge of future demand can materially affect a storage dispatch assessment. Source: Gasper et al., NREL, 2023.

Grid capacity and flexibility control

Measure how accurately the EMS controls power at the grid connection point. Record the magnitude and duration of limit excursions, response time, tracking error and energy delivered against an instruction. Establish the permitted tolerances and response requirements for each scenario before testing.

C&I testing should combine site limits, battery constraints, aggregator requests and commands from the distribution system operator. It should establish which instruction takes priority and how conflicts with building or supervisory control systems are resolved. Residential testing should include connection and phase limits while charging, heating and ordinary household loads operate together. Additional flexibility services are assessed only in markets and configurations where they are available to the customer.

Interoperability

Verify specific combinations of device model, firmware, protocol and use case. A successful test should cover reading measurements, sending an instruction, confirming its physical effect and recovering after a connection failure. Include equipment from multiple manufacturers within a common, published device matrix.

Modbus, MQTT, REST, OCPP and EEBUS describe different interfaces and capabilities. Record the implemented functions and versions. EEBUS distinguishes use cases such as power limitation, dynamic pricing and self-consumption; OCPP certification establishes conformance within the scope of the relevant certificate. Check that scope before attributing a capability to an EMS installation. EEBUS use cases; Open Charge Alliance certification programme.

Reliability and recovery

Test loss of internet access, cloud unavailability, missing or stale meter readings, interrupted device communications, controller restarts and unavailable price data. Record which local functions continue, the fallback state, alarm delivery and the time needed to resume stable control. Recovery should use the actual asset state and prevent old commands from being replayed incorrectly.

For C&I systems, also test the declared number of meters and controlled assets under load. For residential systems, assess whether essential charging and heating remain usable during a service interruption. Islanded backup is tested only where the connected hardware supports it.

Cybersecurity and data governance

Assess device identity, access control, secure configuration, protection of data, software updates and security state reporting. These areas align with the NISTIR 8259 series. Practical checks should include privileged remote access, role separation, update authenticity, logging, vulnerability handling and the supplier's stated security support period. Review data export, retention, deletion and third-party access.

Certificates and compliance statements require supporting documents, including their scope, issuer and validity. ISO/IEC 27001 concerns an information security management system; its scope must be examined alongside product evidence. A claim concerning NIS2 needs an explanation of the applicable organisation, obligations and supporting measures. Neither statement replaces testing the submitted configuration.

Installation, operation and user experience

Give independent installers and users defined tasks and record completion time, errors, assistance required and manual interventions. For C&I, include commissioning, changing a site limit, investigating an alarm and exporting operational data. For residential systems, include setting an EV departure time, adjusting heating preferences, changing tariff details and overriding an automatic schedule. Record which settings require supplier intervention and the resulting delay. Measure missed charging targets and time outside agreed temperature bands throughout the performance tests.

Service, lifecycle terms and portability

Review support hours, contracted response times, documentation, update policy, warranty and end-of-support arrangements. Test the export of historical data and configuration. Establish what continues to work if a subscription ends, an energy supplier or aggregator changes, or the EMS provider discontinues a service. Score clarity and operational continuity here; all monetary costs belong in the operating-value calculation.

Test scenarios for each award

Use a published set of representative installations. Within each scenario, fix the asset capacities, efficiencies, network constraints, initial state of charge, required end state and available information. Separate results by application so that strengths and limitations remain visible.

A reproducible benchmark

1. Establish eligibility and the evidence record

A supplier questionnaire should identify the exact product version, architecture, regional availability, device integrations, control functions, security documentation, pricing and support terms. Record all licence components, including charges per site, asset or installed kilowatt, against the same feature set. Distinguish supplier statements, reviewed documents, witnessed demonstrations and independently reproduced results. An unanswered question is a request for evidence, not proof that a feature is absent.

A broad supplier survey can support screening across hundreds of providers. Apply a published shortlist rule, then subject finalists to the same controlled test programme. State the number surveyed, screened, laboratory-tested and field-validated separately in the eventual results.

2. Define comparable reference cases

Publish separate C&I and residential reference controllers using reasonable, documented rules. Keep the physical assets and service requirements identical when replacing the reference controller with an entrant. This isolates the contribution of the EMS from the value of installing a larger battery or adding solar panels. Test broader equipment compatibility in the interoperability programme.

Use independently recorded load and generation profiles with a common set of country-specific tariff scenarios. Publish the scenario weights, country and contract assumptions before testing. Keep additional market-specific capabilities in clearly identified modules; do not reward every system simply for operating in the country with the most favourable tariff.

3. Control forecasts and test conditions

Use at least twelve months of replay data covering seasonal demand, weather and price variation, with a separate holdout set unavailable during tuning. Deliver only information that would have been available at each decision time. Compare control performance first with common forecasts; assess the supplier's own forecasting in a separate run with equivalent access to historical inputs.

Fix initial and final stored-energy conditions, including battery and thermal storage where relevant, to prevent apparent savings from emptying the installation at the end. Document time resolution, clock alignment, missing-data treatment, meter accuracy and conversion losses. A replay estimates performance under those conditions; it does not establish a year of operational uptime.

4. Validate physical operation

Use a controlled laboratory or hardware-in-the-loop setup, in which the submitted controller interacts with real interfaces and a simulated installation. Validate selected results against physical equipment and field measurements. Repeat critical fault and recovery tests at least three times. Carry out fault injection in an isolated test environment.

Log requested commands, device acknowledgements and independently measured power. Report median and tail response times, tracking error, limit-exceedance duration, unmet service requirements and recovery behaviour. Short field trials validate operation under the observed conditions; describe their length and coverage explicitly.

5. Apply eligibility gates before scoring

Require safe operation within the published limits, a verified fallback strategy, essential security controls and sufficient evidence to reproduce the claimed performance. An unresolved critical safety or security failure prevents an award until corrected and retested under the same rules. Define the severity criteria and permitted tolerances before testing. Cost savings cannot compensate for failure of a mandatory operating or comfort requirement.

6. Score and publish with an audit trail

Score each criterion from 0 to 5 against thresholds published in advance: 0 for verified failure or absence, 1 for major limitations, 2 for partial fulfilment, 3 for meeting the defined benchmark, 4 for a substantial measured improvement and 5 for an exceptional, repeatable improvement across the prescribed cases. Use criterion-specific anchors: security or contractual service evidence requires different thresholds from financial performance. Record missing evidence as “not verified”; do not silently convert it into a pass or redistribute its weight.

Overall score out of 100 = sum of each criterion's weight × its score ÷ 5.

The jury should assess the same evidence dossier, with at least three qualified, unconflicted reviewers per criterion. Use their median score and document significant disagreements. Eligibility for an innovation award requires an innovation score of at least 3 out of 5, supported by demonstrated technical distinctiveness and a reproducible practical benefit. A high overall score alone should not make a conventional product the innovation winner.

All 20 jury members should disclose relevant commercial interests, advisory positions and supplier relationships. Members should withdraw from affected assessments. Publish the applicable rules, test scope, results by criterion and the reasons for the award. Give suppliers the same opportunity to correct factual errors, while preserving the scoring rules. If the difference between finalists is smaller than the observed uncertainty, use a predeclared tie procedure or an additional controlled test.

The resulting report should show where each system performs well, the conditions under which its results were obtained and the evidence supporting its innovation. That gives customers a useful comparison and gives the award a clear technical basis.

View the award categories›