Development Process Overview
ForgeRock's development process utilises the following tool set;
- Software Configuration Management - Git
- Software Configuration Process - Bitbucket Server
- Issue tracking (bugster) - Jira
Individual Developers working on a Feature or Issue
The process overview is shown in the diagram below;
Details on Each Step
This section will walk through each of the steps, using the development of a new feature in OpenIDM as an example. The OpenIDM project home page may be found at https://forgerock.org/openidm, which contains all the links necessary to perform the steps required.
1 - Fork the Central Repository.
Navigate to the central repository for OpenIDM - we can see from the project page that this is found at;
You will need a ForgeRock user account to work with some of the tools. You can create your free account at https://backstage.forgerock.com.
Once on the OpenIDM repository page you can choose to fork the project:
This will create a fork of the central project that is private to you. You can view it's page by clicking on your profile icon (top right of the page), or and choosing view profile. All of your repositories will be shown, including your fork a fork of OpenIDM/openidm.
2 - Create an Issue
All work should be done against a Jira ticket. Jira tickets may be raised at http://bugster.forgerock.org. Choose your project (in this case OpenIDM) For this case, as the new feature we're working on is small and self contained then we will choose the 'New Feature' epic type. If it was more involved and would benefit from being broken down into sub-issues we may choose to create an epic, then 'Story' type issues for each sub-issue. Guides for creating tickets for each of the projects may be found here;
3 - Create a Branch in your Forked Project
In the Jira issue you just created there is a 'Create Branch' link in the Development section. Clicking on this will open a BitBucket server page that will help you create a branch;
Make sure you choose to create the branch in your forked repository! Often the central repository will be selected by default!
In most cases you will want to branch from the master branch.
As you can see, Jira/Bitbucket Server have created a reasonable name for the branch for you, based on the issue number and the title of the issue.
4 - Clone your Forked Repo to your Local Machine
Once you click Create Branch, Bitbucket server will show you the source view of your repo's new branch. From here you can choose to clone the repository to your local machine;
This will present a dialog that will provide you with the url of the repository to clone. Choose the http option from the drop down menu. If you would rather use SSH you will need to set up SSH keys for your account, which can save a lot of typing of usernames and passwords.
If you would rather use SSH you can follow this guide to set up SSH keys for your account. This can save a lot of typing of usernames and passwords
From here you have two options. You can clone the repository to your local machine using either Git on the command line, or by using a git client such as Atlassian's SourceTree. In this tutorial I will use the command line. The screenshot below shows my cloning my fork, then fetching information about all the branches in my remote fork, then showing that information, and finally checkout out the remote branch that JIRA created for me.
5 - Make your Changes
This is where you get creative and write your code! Some points to consider before getting started;
Please follow the Coding Style and Guidelines. There are checkstyle rules configured to test your code against the rules. You may run the checkstyle maven plugin by building with the precommit profile;
mvn clean install -P precommit
There are also xml checkstyle rules for eclipse available from the Coding Style and Guidelines link above.
Some projects have legacy code that fails the newer checkstyle rules. If you're working in one of these modules you should run checkstyle before making changes and compare your checkstyle results after changes to ensure you don't add any errors. Eclipse has a checkstyle plugin that flags up any checkstyle issues as code issues. This can save a lot of time!
Keeping up to Date with Master
Periodically you will want to pull the latest changes made on master into your feature branch. This will avoid a big and nasty merge job at the end of your feature development. To do this you should pull master and then rebase those changes onto your branch.
First get the latest changes from master down onto your machine:
checkout git master git pull origin master
Then checkout your branch and rebase onto master:
checkout bugfix/OPENIDM-5777-further-readme-improvements git rebase -i master
You can learn more about what git rebase is doing on this page. If any exist, you will have to resolve merge conflicts at this point.
ForgeRock have chosen to turn 'fast forward only' on for our public repositories. This makes the master history linear and easy to understand. Compare OpenAM's commit history to say, the linux project's history.
However, this simple history comes at a cost. Re-writing history can get you into trouble if you're not careful. This workflow works really well when only one person is working on a feature or bug branch. The golden rule of rewriting history is that you should NOT do it if anyone else is using that branch. Or rather you need to co-ordinate how you do this with that other person, which works in small teams, but not in distributed teams across different timezones. In that case more complex workflows must be used.
Meaningful and useful commit messages help everyone understand what the commit contains and what it relates too. The Coding Style and Guidelines page contains the following:
When committing code it is essential that others can quickly get an idea of what the commit relates to, and also find more information on the issue and review. A message must always be provided when committing to trunk, sustaining or release branches/tags and we have some simple guidelines for writing them.
- Start with the JIRA issue ID for the story or bug, and the ID of the crucible review for the code.
State in up to 50 chars how this commit changes the product. Begin with a capital letter and don’t end with a full stop. Write as if completing the sentence "If applied, this commit will..."
- If you really need to provide further info in the commit message (info about the fix should be captured in the JIRA issue), then leave a blank line below the summary before adding the details.
Example: AME-9876 CR-1234 Add new authentication module for device auth
Before creating a pull request please build the whole project (don't exclude the unit tests) to ensure you haven't broken anything. At this point you are trying to ensure that you're changes work with the latest code on master. This means you should rebase your branch onto master again, as described in the 'Keeping up to Date with Master' section above.
Then run a full build to make sure nothing has broken:
$ mvn clean install
6 - Push your Commits Back to Your Fork
So you've made your changes, rebased and checked everything builds. It's time to push your changes back to your fork so that you can create a pull request:
git push -f origin bugfix/OPENIDM-5777-further-readme-improvements
We must force push our branch back to our forked repository because in re-writing the history we have made a decision that the history in our local project is the correct history and must 'force' our remote repository's history to look the same. At this point if anyone else is working in a clone of the repository they will find it extremely difficult to integrate their code back into the remote repository. For this reason, for team development, it is better to follow the workflow described in the later section 'Team Contributors'.
7- Create a Pull Request
So, back in Bitbucket Server it's time to create a pull request. A pull request is the mechanism by which you request that another developer (e.g., the project maintainer) pulls a branch from your repository into their repository. In this case, that is done via a review/approval process, and you will perform the final merge into the central repositories's master branch.
To create a Pull Request, browse to Bitbucket's page for the central repo and click the pull request icon in the lefthand vertical menu:
On the pull requests page, click the 'Create Pull Request' button in the top right hand of the page.
Then choose the feature branch in your fork of the repo as the source of the pull request and the central repository's master branch as the destination:
8,9 & 10 Rebase, Push and Merge
Your Pull Request will then be reviewed, which may result in you having to make some changes. If that is the case you can perform your code changes on your feature branch and push them back to your forked repo. The Pull Request will automatically update to reflect the changes.
When two or more reviewers have approved the pull request you're ready to merge. If anyone has merged a PR since you last rebased your branch onto master, you will have to go through another rebase exercise before being allowed to merge. On your local machine:
git checkout master
git pull origin master
git checkout feature/OpenIDM-5821-openidm-s-use-of-j
git rebase -i master
git push -f origin feature/OpenIDM-5821openidm....
After which you will be able to merge your pull request.
Only ForgeRock employees have write access to the central repository of the ForgeRock led projects. This means that once your Pull Request has been approved a ForgeRock employee will do the merge on your behalf.
Due to our fast forward only policy this presents a problem. Only you have access to your private repository branch (that the PR is created from), and only ForgeRock can merge the changes. If you do a rebase on your branch a ForgeRock member will need to merge that Pull Request for you BEFORE another pull request is merged. As soon as another PR is merged then you will need to rebase again before your PR can be merged.
This is awkward and doesn't scale well.
Currently our solution to this is requires that you to give write access to a ForgeRock employee who will then take the following actions;
- Merge Master onto your branch
- Merge your pull request.
This looks odd at first, as the rest of this article focuses on using a rebase strategy rather than a merge strategy. The reason for merging master onto your branch is because it avoids breaking the git commandment 'Do not rebase commits that exist outside your repository' (see here). If we rebased your private repository branch, the clone on your local machine would be effectively broken, and you would have to discard the branch. Merging master onto your branch still allows a fast forward merge but doesn't involve us rewriting the history of your branch. It's your branch after all!
This makes the project history look a little more complex, but is a workable compromise.
Collaborating on a New Feature
The previous workflow used rebasing and rewriting of history, which can cause problems when collaborating on development. Collaboration is better served by a merge workflow.