GSoc/2023/StatusReports/KaranjotSingh

MEASURING ENERGY CONSUMPTION USING REMOTE LAB

The Remote Eco Lab project aims to provide a streamlined process for measuring software energy consumption remotely using a CI/CD pipeline. By automating the measurement process and integrating with the OSCAR tool, developers can make informed decisions to improve code efficiency and work towards software eco-certification

Mentors

Volker Krause

Benson muite

Nicolas Fella

Merge Request

The official repository for the project can be found at: https://invent.kde.org/teams/eco/remote-eco-lab

Due to the extensive need for Continuous Integration (CI) testing and CI-related modifications, the majority of the work has been carried out in this test repository: https://invent.kde.org/drquark/kecolabtestci

The majority of the alterations related to CI testing can be found detailed in: https://invent.kde.org/drquark/kecolabtestci/-/merge_requests/9

For accessing the most recent artifacts and the final report, please visit: https://invent.kde.org/drquark/kecolabtestci/-/pipelines/457169

Blog Posts

Google Summer of Code : KEcolab

Timeline

Week 1 and Week 2

During the community bonding phase, we established a communication channel (accessible at https://matrix.to/#/#kde-eco-dev:kde.org) to encourage transparent collaboration and maintain a public record of our discussions. This channel also served as a means to engage additional individuals in the project.

Furthermore, we organized regular internal meetings, occurring weekly on Thursdays, to discuss project updates within our team. Additionally, we presented project progress to the wider community during our monthly meetups.

In the first and second weeks, with guidance from our mentors, we successfully set up the project infrastructure, which involved tasks such as configuring SSH keys, setting up Raspberry Pi1, and preparing the lab test PC. My specific contribution included configuring the X server on the lab test PC to enable the execution of graphical user interface (GUI) applications. During this timeframe, I started work on implementing the first stage ie installation of Flatpak as part of the Continuous Integration (CI) Pipeline.

I also authored a blog post to showcase our project to the broader KDE community.

Week 3 and Week 4

I successfully finished the implementation of the Flatpak installation phase. Over the course of these two weeks, I also began tackling the energy measurement aspect of the CI Pipeline. However, I encountered some challenges related to copying files (via scp) from the repository to the lab test PC and configuring the necessary environment variables. With the help from the mentors, I was able to resolve these issues. The solutions primarily involved executing all the commands within a single session, simplifying the process.

As we reached the end of the fourth week, I had achieved the ability to execute and complete the energy measurement stage in the pipeline. This stage allowed us to generate artifacts, including power meter readings and hardware data, which are subsequently passed to the next stage for analysis and report generation.

Week 5 and Week 6

I began my work on the final phase of the measurement pipeline, which is the result stage. This phase demanded that I acquire proficiency in the R programming language. Thus, during my fifth week, I dedicated time to learning R and subsequently focused on creating preprocessing scripts tailored for the OSCAR tool(Open-source Software Consumption Analysis in R, was developed by students from Umwelt-Campus Birkenfeld). It's worth noting that OSCAR presents significant challenges, particularly in the area of data formatting. To address these formatting complexities, it became imperative to preprocess the scripts before feeding them into OSCAR.

Within the preprocessing script, my primary tasks revolved around handling user input as script arguments and performing conversions from epoch time, all while ensuring that date and time were correctly formatted for OSCAR.

Additionally, I attempted to install OSCAR and R on our Raspberry Pi. However, this endeavor proved impractical due to the substantial resource requirements, including issues related to dependencies and available RAM. Consequently, after talking with mentors, I made the decision to utilize the GitLab CI pipeline for both the installation and execution of OSCAR.

Week 7 and Week 8

During this week, I made changes to the OSCAR script so that it could be used in the GitLab CI Pipeline. This mainly involved fixing problems with how data was organized and presented. For example, there was an issue with the way dates were formatted - the format "DD-mm-yy" is different from "dd-mm-yy," and this was causing errors in the final results. To fix this, I carefully examined the code line by line and collaborated with Achim, who developed OSCAR, to resolve these issues. These issues were important because they caused the report to have "NA" (not available) values in various columns, making it hard to create a proper report file.

I also tackled a problem where OSCAR couldn't convert the date and time into the POSIX time format, resulting in null timestamps. An important thing to note is that if any value becomes null, it causes all columns and rows to have a value of 0. This problem was similar to the one above related to date formatting.

After dealing with rendering and debugging errors, I successfully managed to generate the report by the end of the eighth week. This report was initially created for the already tested tool Okular, but I adjusted the script to generate reports for any application, and I successfully generated one for Kate as well.

Week 9 and Week 10

So after testing the OSCAR R script locally on my system, I wrote the result stage for Gitlab CI pipeline and tested it using a tidyverse image which already had most of the dependencies installed and is much lightweight. The script uses the updated OSCAR tool from the official repo, it takes the artifacts from the previous stage and ran the OSCAR tool with the updated artifacts to generate the report. I encountered an issue regarding automating the date on the filename so that it would be able to generate the artifact. The collectl output saves the file as `test1.csv-dd-mm-yy.tab` so to download the file from the system i needed to know the current date and time and replace it with that. As i needed to handle it with both the ways from the test lab system as well as the gitlab CI pipeline. I defined variables on both the end, In the test lab system i copied the file into a new generic name `test1.csv` and on the gitlab pipeline i used a variable to export it as a artifact with a name including the date and time `test1.csv-dd-mm-yy` ( Also we require this format of the file because the preprocessing scripts takes the date and time for preprocessing from the filename. I also tried regex to successfully export the artifact like this `"test1.csv-joseph-esprimop957-*.tab.gz"`. Another issue was with the forwarding the artifacts from one job to another job which was solved by adding the dependencies field in the next job and by first copying the artifacts into the required directory ( as if we `cd` into any directory, we basically lose the default directory where the artifacts are placed ). Both of these issues were solved with the help of mentors. After this, All the three jobs ran successfully and i was able to generate an energy measurement report.

Week 11 and Week 12

For now i am migrating the code from test repository to the official remote eco lab repository and currently the work is in the progress of adding gitlab custom runners which requires help with the sysadmin side of KDE. Also I am working on documenting my work and writing a good readme for the future testers and contributors on the official repo.