Convert homegrown unit tests to use the Clar framework

Hi Guys, In this blog, I want to discuss what I’ll be working on during my internship with Git and roughly how I would go about it.

The major object of the internship task is to convert the unit tests found in the t/unit-tests folder using the homegrown testing framework t/unit-tests/test-lib.h to use the clar testing framework: https://github.com/clar-test/clar

The existing structure of Git’s unit tests consists mainly of .c files within t/unit-tests/, leveraging custom frameworks and assertions. There are 27 files in the t/unit-tests folder, two of which (strvec.c & ctype.c) have been converted to use Clar by one of my mentors, Patrick Steinhardt. There are currently 21 test scripts prefixed with the t-“script name” that need converting.

Why the need for this conversion?

Clar is a lightweight, simple, and flexible testing framework primarily designed for C programs. It is particularly used in projects like libgit2, providing a clean and minimalistic approach to writing and organizing tests. Below are some key benefits of using Clar:

  • Simplicity: Clar is designed to be straightforward, making it easy for developers to write and maintain tests. Its minimalistic approach reduces the complexity.

  • Flexibility: Despite its simplicity, Clar is quite flexible. It allows tests to be structured in a way that suits specific needs, making it adaptable to testing scenarios.

  • Speed: Clar is optimized for speed, which makes it easy for continuous integration and frequent testing. Its lightweight nature ensures that tests run quickly.

  • Focus on C: Clar is specifically designed for C programs; it integrates well with C projects.

  • Easy to Integrate: Integrating Clar into a project is straightforward. Its minimal dependencies and simple setup process make it easy to start using without significant modifications to the existing codebase.

  • Custom Assertions: Clar allows for the creation of custom assertions tailored to the specific needs of a project.

Benefits Observed previous conversion

Before converting the unit test to use clar, each test function needs to be declared and connected to the main function (entry point), which causes repetition and is quite cumbersome to implement for every test function. This issue drove the discussion for adopting a more standard framework.

As displayed in this patch: link

The following improvements can be observed:

  • Clar’s test structure provides a more organized, descriptive test functions, such as test_strvec__push or test_strvec__clear. Each function isolates a specific test scenario, enhancing readability and maintainability.

  • Clar’s macros, such as cl_assert and cl_assert_equal_, provide uniform assertion syntax, making tests easier to understand and maintain. The check_ functions from the previous framework were custom, adding complexity, while Clar’s built-in assertions improve standardization.

  • Using Clar aligns Git’s unit tests with a more structured framework, facilitating integration across the project. It potentially simplifies contributions, as developers familiar with Clar can readily understand and add tests.

  • Clar includes built-in support for more complex test setups and tear-down functions, making it easier to expand testing scenarios if new strvec features are introduced.

  • By adopting Clar, there’s an opportunity to identify and address any framework limitations through upstream contributions, potentially improving Clar itself and benefiting future Git projects.

Project plan

  •  Initial Preparation and Codebase Familiarization
  •  Develop a structured approach to incremental conversion of tests.
  •  Initial Conversions and Incremental Submission
  •  Feedback and Clar Collaboration
  •  Continued Conversions and Feature Integration
  •  Final Documentation and Wrap-Up

So far

I’m in the first phase of my project plan, studying the current test structures and how the entire testing system works. I also plan on not limiting my scope to just the test structures. I intend to familiarize myself with the entire codebase and file structure.

Also, I had to set up this blog using the Jekyll templates on GitLab Pages, my first time setting it up. I encountered some issues, which are stated below:

  • Setup: So I made a little mistake setting up the GitLab Jekyll pages. I used a past intern’s page setup to build my site, but I didn’t realize that the Jekyll templates had different flavors. The page I was trying to mimic had the Scribble flavor of Jekyll setup while I had installed the native template.

    It took a little while before I could figure it out and fix the issues I had trying to treat the template as the same thing. I, however, still stuck with the native template and made it work. Honestly, the setup is not the best, but it works, haha!

  • Familiarity: Trying to understand how things work with Jekyll was a bit tricky. The file structure system and how written posts in a markdown file translate into the site pages that are built-in HTML files took a bit of digging into.

    My knowledge of everything is still a work in progress, but so far I’m enjoying the learning journey.

Next Steps

After I’ve done sufficient familiarization with the codebase, I’ll commence prioritizing the test scripts on their level of complexity. Easy test scripts will be converted and progressively into the more difficult ones. I’ll be sharing more of Journey in my subsequent post. See you soon!