GSoc/2023/StatusReports/KaranjotSingh

MEASURING ENERGY CONSUMPTION USING REMOTE LAB

The Remote Eco Lab project aims to provide a streamlined process for measuring software energy consumption remotely using a CI/CD pipeline. By automating the measurement process and integrating with the OSCAR tool, developers can make informed decisions to improve code efficiency and work towards software eco-certification

Mentors

Volker Krause

Benson muite

Nicolas Fella

Merge Request

The official repository for the project can be found at: https://invent.kde.org/teams/eco/remote-eco-lab

Due to the extensive need for Continuous Integration (CI) testing and CI-related modifications, the majority of the work has been carried out in this test repository: https://invent.kde.org/drquark/kecolabtestci

The majority of the alterations related to CI testing can be found detailed in: https://invent.kde.org/drquark/kecolabtestci/-/merge_requests/9

For accessing the most recent artifacts and the final report, please visit: https://invent.kde.org/drquark/kecolabtestci/-/pipelines/457169

Blog Posts

Google Summer of Code : KEcolab

Timeline

Week 1 and Week 2

During the community bonding phase, we established a communication channel (accessible at https://matrix.to/#/#kde-eco-dev:kde.org) to encourage transparent collaboration and maintain a public record of our discussions. This channel also served as a means to engage additional individuals in the project.

Furthermore, we organized regular internal meetings, occurring weekly on Thursdays, to discuss project updates within our team. Additionally, we presented project progress to the wider community during our monthly meetups.

In the first and second weeks, with guidance from our mentors, we successfully set up the project infrastructure, which involved tasks such as configuring SSH keys, setting up Raspberry Pi1, and preparing the lab test PC. My specific contribution included configuring the X server on the lab test PC to enable the execution of graphical user interface (GUI) applications. During this timeframe, I started work on implementing the first stage ie installation of Flatpak as part of the Continuous Integration (CI) Pipeline.

I also authored a blog post to showcase our project to the broader KDE community.

Week 3 and Week 4

I successfully finished the implementation of the Flatpak installation phase. Over the course of these two weeks, I also began tackling the energy measurement aspect of the CI Pipeline. However, I encountered some challenges related to copying files (via scp) from the repository to the lab test PC and configuring the necessary environment variables. With the help from the mentors, I was able to resolve these issues. The solutions primarily involved executing all the commands within a single session, simplifying the process. As we reached the end of the fourth week, I had achieved the ability to execute and complete the energy measurement stage in the pipeline. This stage allowed us to generate artifacts, including power meter readings and hardware data, which are subsequently passed to the next stage for analysis and report generation.

Week 5 and Week 6

I started working on the last stage of the measurement pipeline i.e the result stage. It required me to learn R language so during my 5th week, i learned about using R and then worked upon writing preprocessing scripts for the OSCAR tool (Open source Software Consumption Analysis in R developed by Umwelt-Campus Birkenfeld students). OSCAR is very challenging tool to work with when it comes to formatting so to fix the formatting issue, it was necessary to preprocess the scripts before providing it to OSCAR. In the Preprocessing script, I mainly worked upon taking arguments from the user on the script and also converting from epoch time and formatting the date and time for OSCAR. Also I tried installing OSCAR and R on our Rasberry PI but as the process requires a lot of resources (Dependency and RAM issues) so it wasn't suitable so after discussing this with the mentors i settled of using the gitlab CI pipeline for installing and running OSCAR.

Week 7 and Week 8

During this week, I modified the OSCAR script for using it on the gitlab CI Pipeline. This involved fixing issues related to formatting for eg. There was an issue with formatting of date ie the format DD-mm-yy is different from dd-mm-yy and this was causing an error in the final result. I debugged the code line by line and took help from Achim (Developed OSCAR) to fix issues as most of these issues cause the report to has a NA in various columns thus making it difficult to render it into a report file. I also solved an issue regarding oscar not being able to convert the date and time into the POSIX time which resulted the timestamp to be Null and one thing that is worth noticing is the fact, if any one of the value gets to be NULL it results in 0 value in every column and row, This issues was similar to above one on the formatting of date. Finally after some errors with rendering and debugging it, i successfully was able to generate the report by the end of week 8. This report was generated for already tested tool Okular so I modified the script for a general input to generate report for any application and i was able to generate it for Kate

Week 9 and Week 10

So after testing the OSCAR R script locally on my system, I wrote the result stage for Gitlab CI pipeline and tested it using a tidyverse image which already had most of the dependencies installed and is much lightweight. The script uses the updated OSCAR tool from the official repo, it takes the artifacts from the previous stage and ran the OSCAR tool with the updated artifacts to generate the report. I encountered an issue regarding automating the date on the filename so that it would be able to generate the artifact. The collectl output saves the file as `test1.csv-dd-mm-yy.tab` so to download the file from the system i needed to know the current date and time and replace it with that. As i needed to handle it with both the ways from the test lab system as well as the gitlab CI pipeline. I defined variables on both the end, In the test lab system i copied the file into a new generic name `test1.csv` and on the gitlab pipeline i used a variable to export it as a artifact with a name including the date and time `test1.csv-dd-mm-yy` ( Also we require this format of the file because the preprocessing scripts takes the date and time for preprocessing from the filename. I also tried regex to successfully export the artifact like this `"test1.csv-joseph-esprimop957-*.tab.gz"`. Another issue was with the forwarding the artifacts from one job to another job which was solved by adding the dependencies field in the next job and by first copying the artifacts into the required directory ( as if we `cd` into any directory, we basically lose the default directory where the artifacts are placed ). Both of these issues were solved with the help of mentors. After this, All the three jobs ran successfully and i was able to generate an energy measurement report.

Week 11 and Week 12

For now i am migrating the code from test repository to the official remote eco lab repository and currently the work is in the progress of adding gitlab custom runners which requires help with the sysadmin side of KDE. Also I am working on documenting my work and writing a good readme for the future testers and contributors on the official repo.