Starting Sprint 6.2 I will have limited capacity, this means that the overall teams capacity for the sprint will be lowered. Capacity is based on the team’s available hours for the sprint, and the goal Is to try to fill the capacity effectively without overburdening and under committing team members. For example, for Sprint 6.2 we have an available capacity of 10 with a reserve capacity of 4. This means that we have a max estimate of 10 points between all team members to use on completing tasks within the sprint. These points can be split up in various ways, and usually Is determined by priority of a task within the backlog. There are an additional 4 points of reserve capacity that are set aside for meetings, peer review, etc. that could be used toward completing another task If time and workloads permit. Reviewing these capacities Is an Important part of PI Planning and Sprint Planning, when planning PI 6 I Informed the team I would have a capacity of 2 for each sprint except for Sprint 6.2 where I would have a capacity of 1. PI 6 Planning doc(Image 1) contains more Information for future sprints as well as past sprints.
Image 1 - PI 6 Planning
What Is the task? And the description of the task
Task: As a developer, I need a reusable templating class that can be used to enable a variety of templating needs
Description:
Create and contribute a Template Class that matches that outlined in the Template Tool Design Doc
A Template class scaffolding has been included here. All methods and functionality should be filled in
The work of hooking this Template Class into the existing Namelist Handler Tool will be done as part of UW-192
The templater.py currently does a lot of this work, but in its "main" function
Tests for this Class and its methods can be written and committed in tests/test_template.py
What does this mean?
From a quick read of the description I had several follow-up questions:
What does It mean by a reusable template?
It seems as though we'll be editing the Template class scaffolding provided to us. Are there any example files we can model our file off of?
What is the difference between this new Template class and what templater.py does currently?
Does this ticket consist of writing the Template Class and Unit Tests for the class?
Accomplishing the Task
For ticket 191 there were a lot of resources that needed to be sifted through before addressing the task. This Involved going through template.py, which currently did a lot of this work and understanding why we were shifting away from this code. Accomplishing UW-191 became a bigger task than we previously Imagined. We first needed to understand UW-191 and set-up the project.
After being stumped on where to begin with UW-191, Emily and I decided to Inform the team In our Tuesday DSM that we were blocked due to not knowing where to begin with the task. There were many resources on the task that we needed to utilize to edit the template.py file, which would house our code for the reusable template class. Without understanding the full scope of the project we weren't able to continue. We were able to address our concerns and decide to use our Weekly Tagup Meeting with Christina to get clarity on our blockers.
In the Tagup, Christina gave us valuable insight into which files were important for us to go through, understand and be able to reuse snippet of code within template.py.
Breakdown:
We had three different template files: templater.py, template.py, and J2template.py (Images 2 - 4)
templater.py - previously had already done most of the work in a main function, but we needed to change it to incorporate Jinja so if the system changed it wouldn’t break the code
template.py - our new file to add our reusable template class code to
J2template.py - Terry created this file, that Included code examples for us to model our template class off of
After more research with Christina, we came to the conclusion that J2template.py contained the exact code we needed to write for template.py (look below at Images 3 & 4, and you'll see no differences between the comments). Knowing this we were tasked to confirm Terry's code followed the guidelines of UW-191's acceptance criteria, and would be able to skip writing the reusable template class If It did, then we would be able to write additional test cases to test this code in test_template.py.
Setting up our testing environment:
Christina was able to walk us through the process of using conda to run tests locally In our command line. Conda is an 'open source package and environment management system that allows quick installs, runs and updates packages and their dependencies. Conda easily creates, saves, loads and switches between environments on your local computer.' We were able to create a conda environment called wf_tools_test.
Now we could clone the workflow-tools repository that contained all the code for the Unified Workflow Team, and run the tests from the command line locally.
Overall we were able to leave our Weekly Tagup meeting feeling more confident than before In accomplishing UW-191.
Image 2 - templater.py
Image 3 - template.py
Image 4 - J2template.py
Next Emily and I were deciding between using VSCode or Vim for our IDE. An IDE is 'a software application that helps programmers develop software code efficiently.' IDE stands for integrated development environment. Vim is 'technically not an IDE, but is the most popular text editor used by developers today.'
We decided to use Vim for our editor rather than using VSCode because Christina would be able to give us support since she uses the application as well. We also found it easier since Emily and I were previously familiar with the application from our Software Tools class. This gave us an advantage rather than trying to learn the basics of a new application/tool with only a week left in the sprint.
We also were able to look through Terry's code and confirm that it followed the guidelines of UW-191's acceptance criteria, and matched the outline in the Template Tool Design Doc. Now we could write additional test cases in test_template.py.
Emily and I found a recurring theme from Sprint 6.2, where we would be In a meeting and feel like we understood the vision and the Inner workings of 191 much more clearer, but when we went to pair program on It we couldn't make much progress. Due to this we set-up an additional Tagup meeting with Christina for Monday before our weekly Tuesday Tagup. In this extra meeting we were able to gain more clarity around what the test cases should look like. Christina was able to provide us with sample code that hopefully would get us over that last hurdle In understanding 191.
Sample Code:
Hi there {{ name }}
J2Template(template_str="Hi there {{name}}")
J2Template(template_path="/path/to/test/file")
test_config = {'name': 'Christina'}
mytemplate = J2Template(test_config, template_str="Hi there {{name}}")
assert "Hi there Christina" == mytemplate.render_template()
With less than one week left In the sprint and no code to show for our efforts I think we both began to feel discouraged. We decided before our normally scheduled tagup meeting we would try to write some example test cases to get feedback from Christina to see If we were going In the right direction. Within j2template.py we had seven functions:
__init__(self, configure_obj, template_path=None, template_str=None)
dump_file(self, output_path)
_load_file(self, template_path)
_load_string(self, template_str)
render_template(self)
undeclared_variables(self)
validate_config(self)
Starting with dump_file, _load_file, and _load_string was the easiest for us because they were the simplest functions and had the least amount of parameters. A parameter Is 'a variable in a function definition, It is a placeholder and hence does not have a concrete value.' For example, self, output_path and template_str are all parameters within the function definitions listed above. After writing a couple basic test cases we were able to run them locally, but were getting too many errors to narrow down the problem. We decided to wait for our weekly tagup to get feedback from Christina.
Showing Christina our test cases, she was able to point us In the right direction. Once you close out of a terminal window on your local computer, you have to reactivate your conda environment using the line 'conda activate <environment>', our environment was set to 'wf_tools_test,' however we had not reactivated our environment In the terminal. Following the reactivation of the environment, we were able to see that 1/3 of our test cases passed. We were also given more helpful errors output by the terminal to debug the other two test cases, and within 10 mins had three working test cases(Images 5 - 7 highlight some of the errors we saw). Reviewing our tests we had already tested the __Init__ and render_template functions as well. This meant that If we could test undeclared_variables and validate_config we would have accomplished UW-191.
Feedback:
Gaining some more Information from Christina, she asked us to remove the code for validate_config completely from j2template.py, allowing us to skip a test case for this function. We also were informed that passing the pytests In the command line weren't the only test we needed to run, we also needed to pass the pylint tests for uwtools and tests.
What is Pylint? Pylint doesn't run your code, but checks it and enforces a coding standard. 'It can tell you the issues in your program and also, a way to resolve them, ultimately giving you feedback on your code.' Essentially, Pylint would confirm the code we wrote In j2template.py and test_template.py was up to PEP-8 standards.
With that knowledge, we began working on testing undeclared_vairables and running both sets of test cases - pytest and pylint.
Images 5 - 7
Closing out UW-191 we needed to push our code to the workflow-tools repository on GitHub. It's standard not to push your code directly to the main branch - In our case the develop branch - but to create a new branch locally and then push your code to GitHub. It is recommend to do this because It allows you to isolate your changes from the main branch. We also could only push our changes to the repository once our code passed all the pytest cases and we received a 10/10 feedback rating for the pylint for both uwtools and tests. After running the pylint we saw quite a bit of feedback where our code needed to be revised to meet the workflow-tools standards (Image 8).
Image 8 - Pylint tests output feedback
Reviewing the feedback, we saw that these were just basic updates such as removing additional spaces from the ends of lines, correcting the Indentation level from 3 spaces to 4, and others. This updating didn't take long, and we were able to see 10/10 marks for both pylint tests and uwtools(Images 9 & 10). We were also able to pass the pytests(Image 11).
Image 9 - Pylint uwtools pass
Image 10 - Pylint tests pass
Image 11 - pytest pass
With all of the tests now passing, we were able to push to GitHub and create a Pull Request(Image 12). A Pull Request(PR) notifies your team that you have pushed your changes to a branch and are awaiting confirmation that you can merge your code Into the main branch(develop branch In our case). We were able to get our Pull Request approved by the team and Fredrick merged our branch Into the develop branch on Friday, the last day of Sprint 6.2.
Reflection:
I think this sprint showed a pattern of constantly reviewing the resources given to us, asking more questions, meeting with Christina, and reestablishing our understanding of the objective. The idea of review, question and reestablishment was the key takeaway from this sprint hence the name of this weeks blog post. Overall, It felt very rewarding to experience the whole process of development and merge a PR this early in a PI. Christina was a great help on this sprint by making time to meet with us for additional meetings, which helped tremendously In completing UW-191.
Image 12 - GitHub Pull Request workflow-tools Repository
Resources referenced to complete UW-191:
https://github.com/ufs-community/workflow-tools/blob/develop/tests/test_config.py (line 63: def test_f90nml_config_simple)
https://github.com/ufs-community/workflow-tools/blob/code_examples_UW-191/src/uwtools/j2template.py
https://alexharv074.github.io/2020/01/18/a-method-of-unit-testing-jinja2-templates.html
https://stackoverflow.com/questions/70409343/assert-to-check-if-a-element-present-in-a-list-or-not
https://stackoverflow.com/questions/59041955/python-typeerror-set-object-is-not-subscriptable
https://medium.com/swlh/python-coding-tip-using-the-with-statement-instead-try-finally-f45a645c6008
To end Sprint 6.2, we close out with our Sprint Planning and Retrospective meeting. Again, in this meeting we covered what went well, what could've been better and discussed more on the sprint ahead. Image 13 highlights an overview of the meeting.
Image 13 - Sprint 6.2 Retrospective
Sprint 6.3 will begin Monday, October 10th and next week I will discuss which task we have been assigned for the sprint.