SEM impact evaluations are a crucial part of ensuring programs operate with the highest level of excellence. However, an examination of recently published SEM impact evaluations revealed consistent recommendations for better data management, measurement, verification, and project documentation. Providing these improvements means streamlining SEM program evaluation with less time and resources needed to establish important program metrics like gross savings and effective useful life (EUL).
Cascade’s energy management software, Energy Sensei, helps program administrators, evaluators, and implementers collaborate effectively through a central, online platform. Consequently, the outcome is lower evaluation costs for any size SEM program. Additionally, eradicating missing information lowers uncertainty, greatly benefiting all parties.
For program managers, using Energy Sensei also helps reduces costs and time. Moreover, evaluators need a lot of information to complete their job. Energy Sensei centralizes the needed information, enabling evaluators to get that data with less lift. Additionally, as a SOC II Type 2 certified software, all program data is secure. Say goodbye to versioning issues, aggregation errors, and misfiled reports. Instead, Energy Sensei provides evaluators access to only specified data, meaning pre-loaded analysis views are easy to work with and require less effort to rebuild themselves.
Recommendations from recent evaluations for ComEd (2022), Energy Trust of Oregon (2019), California Public Utility Council (2021), and Bonneville Power Administration (2020) point to several critical needs for effective evaluation:
Let’s talk through some examples of how Energy Sensei helps evaluators and program managers work together in streamlining SEM program evaluation processes.
Accepted best practices recommend providing the finest interval data available. Yet, aggregation errors are a common source of discrepancy in evaluation. As programs scale up, manual management of data streams becomes more challenging to manage, with more potential for aggregation or meter to meter error.
Energy Sensei accepts fine interval data and automatically aggregates it to the level needed for measurement and verification. Automated data feeds through Green Button and secure file transfer protocols (sFTP) ensure data is attributed to the right meter for each participant, yielding clean datasets and less manual data handling. Additionally, automating data also facilitates extending data collection in preparation for persistence studies.
Whether the data is energy, occupancy, or production, it can be visualized, tracked, and reviewed in the Sensei platform. The platform also provides temperature data, taking that data stream off everyone’s plate.
If you’ve ever wondered, “Is this the right version?” when looking at an opportunity register, Sensei is for you. A program, cohort, or site’s opportunity register can be viewed with the latest updates. Energy teams and coaches update the register during regular check-ins, leading to more complete information.
Here are a few key evaluation-specific benefits of Sensei’s register:
Opportunity register information is easily filterable and can be reviewed within the platform or downloaded for more detailed analysis.
Speaking of inflections, Sensei empowers users to flag events for further investigation. Program implementers and SEM participants can create ‘Events’. Each Event includes a date range and creates a flag on timeseries charts. Consequently, this promotes early discussion between SEM participants and coaching teams on changes in energy intensity.
This capability is key in streamlining SEM program evaluation. Several recent evaluations (SCE, ComEd) encountered non-routine events that could have been flagged and discussed during the program. Sensei helps find these with ‘Data Inspector’ which identifies outlier residuals and data points. As a result, post-project non-routine event adjustments were among the most impactful differences between evaluation and program reported savings results. Sensei provides tools to better identify NREs and to document the whole process.
Energy Sensei also generates energy models using temperature and other energy driver data streams. Hypothesis models are created, tested, and honed within the software. Standard statistics for model fitness are automatically generated and provided along with critical information like date ranges, historical data, and acceptable variable ranges. Energy, temperature, and dependent variables are visualized in time series and x-y plots. Additionally, the viewer can customize the interval viewed to help better identify atypical data.
Sensei tools include outlier inspection and consistent documentation and handling of missing, excluded, adjusted, or non-programmatic savings values. Evaluators can view all of this information within Sensei. If additional analysis is needed information can be provided in a consistent, transparent manner via a bulk database export (program-wide analysis) or project-specific workbooks (deep dive on individual projects).
Savings claims are tracked within Energy Sensei. Using Sensei as the claim record reduces errors and provides a traceable pathway to verify both top-down and bottom-up savings. Furthermore, each claim includes additional detail of dates and the method used to arrive at the savings value.
Energy Sensei is a robust platform any program can use to standardize their reporting and delivery. Program implementers all use the same platform and tools to deliver and report results. Savings, measures, and other key metrics can be viewed at a participant, cohort, or program-level, giving you better access to your program potential and progress. As a result, updates made by coaches and participants take effect immediately, so you’re always seeing the most up-to-date information.
SEM impact evaluators serve an important role as we all seek to bring excellence to clients and programs. Reliable updating, raw data management and elevated modeling are all recommended elements of improvement. Striving for these together is how we achieve collective wins. Let Energy Sensei do the heavy lifting in elevating both SEM implementation and streamlining SEM program evaluation. Freeing your team from logistics allows you to fully focus on your goal: saving energy.
Reach out here to learn more about Energy Sensei!