I have used many open source products in my professional life. I have not studied the sources of many of these products and projects. My main contributions to the open source community consisted of presentations (many dozens) and articles (many 100s) in which I have spread the word. I have created workshops and demo applications and shared those on GitHub and my blog. I have reported an issue in Struts (2003) and last year I have submitted two Pull Requests for faker,.js.
Today it is time for a more substantial contribution. After some deliberations – do I have the required skills and knowledge to create the code, do I know how to construct my contribution in the acceptable way, do I know how to actually submit my contribution – I have decided that I should be able to go ahead and create and submit a custom state store component to the Dapr project. I am not the best programmer in the world – but I should be able to do a solid enough piece of programming that does the job and is good enough to be shared and reviewed. Although putting my code out there is a little scary, if I follow the guidelines and the process diligently it should be a good experience. The review may reveal some weakness and shortcomings in my code – but surely I will learn from any feedback? Open source products – and commercial, closed source products too – are created after all by people like me. Enthusiasts, programmers and perhaps sometimes dreamers. Why would my contribution not be of value?
I have mentioned the project I am going to submit a contribution to: Dapr – see Dapr.io. Dapr is the Distributed Application Runtime, a framework that provides APIs that simplify microservice connectivity and that make it tremendously easy for application to leverage a number of generic services: state management, service discovery, message broker integration, encryption, observability, and secret management, Applications that are “daprized” get a personal assistant (Dapr) that can handle many tasks for the application.
This article is by no means intended as an introduction of Dapr. There are many other resources that do that very eloquently and clearly. One of the tasks Dapr can handle is state management: applications can ask Dapr to save state and to retrieve state.
The application code itself stays stateless yet the application can benefit from having its own state all the same. Dapr can use various products and cloud services as its backend state store implementation. Dapr supports for example Redis, Azure Blob Storage, Azure CosmosDB, Couchbase, MySQL, Zookeeper, Apache Cassandra, (see this full list of supported state stores). Dapr is open: it has clearly defined APIs and a clear mechanism for adding additional implementations of state stores (or one of the other component types).
One cloud service that is currently not supported by a Dapr state store component is the OCI ObjectStorage service on Oracle Cloud Infrastructure – note that currently none of the OCI services is supported through a Dapr component. I believe that for Daprized applications running on OCI – either on OKE, the OCI container engine for Kubernetes or on a compute instance – it can be of value to leverage ObjectStorage as the state store. It is cheap, easily accessible and can easily be integrated into many technologies from within the OCI cloud as well as from elsewhere. At the same time, creating a Dapr StateStore component for OCI ObjectStorage is a great exercise and learning experience for me to get to learn Dapr a little bit better, get some hands on experience with Go, participate in an open source project and help promote the use of Oracle Cloud Infrastructure services for my current and future colleagues and customers.
Preparing the Dapr State Store Component based on OCI ObjectStorage Service
In several recent articles, I have shown in detail:
- How to create a local environment for building and running a custom version of Dapr: Customizing Dapr-preparing my environment
- How to prepare the local Dapr environment for customizing compoents or even creating custom component (and hooking them into the Dapr runtime): Extending Local Dapr Development Environment for Custom Components
- How to create a Custom Dapr State Store Component based on OCI Object Storage: Composing Dapr Custom State Store Component for Oracle Cloud Object Storage
With these articles, I have reached the point where I have working code for the new State Store component and I have it hooked into my local Dapr environment. I have created a fairly extensive set of unit tests for the component. I feel as if I am ready to share my component with the Dapr code reviewers. However, I do feel a little bit anxious and I want to do things right. To make sure the people reviewing and potentially accepting my submission do not have to put in more effort than is really necessary.
An important document at this stage is this page on the dapr.io project site: Contribution overview – General guidance for contributing to any of the Dapr project repositories ( https://docs.dapr.io/contributing/contributing-overview/); more specifically geared towards development of a custom component – my personal ambition – is this page: Developing New component in the components-contrib repository (https://github.com/dapr/components-contrib/blob/master/docs/developing-component.md). I have followed the instructions under Write New Component and Validating with Dapr Core. The next section is relevant now: Submit your Component.
The steps listed are:
- Create a pullrequest to add your component in component-contrib repo
Note: Make sure there’s an issue (bug or proposal) raised, which sets the expectations for the contribution you are about to make. - Get the approval from maintainers
And only after the approval has been received I can continue with: Fetch the latest dapr/dapr repo, Update component-contrib go mod and ensure that component-contrib is updated to the latest version, Import your component to Dapr main.go, Register your component in Dapr main.go and finally Create a pullrequest in Dapr.
For now I first have to:
- Make sure there’s an issue (of type proposal) raised, which sets the expectations for the contribution you are about to make.
- Fork the components-contrib repository
- Clone the repository locally
- Create a branch for re-creating the custom state store component and its tests
- Create code – component implementation and unit tests
- Commit the new component’s sources – Commit with DCO sign-off in the commit message
- Update relevant documentation for the change (or at least create an issue in the Dapr documentation repository for this to happen)
- Create a pullrequest in the Dapr components-contrib repository from my branch
The next figure illustrates these steps – that involve the public GitHub Dapr repositories compnonents-contrib and docs, forked repositories and my local development environment.
Let’s go through these steps:
1. Issue – Proposal for OCI ObjectStorage State Component in Dapr repo
I have created an Issue of type Proposal in the Dapr components-contrib repository that describes the new State Store: https://github.com/dapr/components-contrib/issues/1400
In the PR I will refer to this proposal, similar to what akkie did with this issue – proposal for Zeebe.
After just a few hours I received a very encouraging reaction to my issue:
2. Fork the Dapr components-contrib repository
Go to https://github.com/dapr/components-contrib and click on Fork.
Indicate where the fork should be created. Next, the forked repo is opened.
3. Create a local clone of the forked components-contrib repository
Clone the forked repository locally.
git clone https://github.com/lucasjellema/components-contrib
4. Check out on branch oci-objectstorage-state
Create a new branch called oci-objectstorage-state and switch to that branch:
git branch oci-objectstorage-state
git checkout oci-objectstorage-state
5. Create code – component implementation and unit tests
The code I created earlier – as described in this article – can now be copied into this directory structure:
cp -R ~/dapr-dev/components-contrib/state/oci ./state/
Final check of the unit tests:
make test
or:
go test github.com/dapr/components-contrib/state/oci/objectstorage -v
Final Linting check:
golangci-lint run ./state/oci/objectstorage/objectstorage.go
That unfortunately revealed a fairly long list of violations and caused me to fix and improve the code quite considerable.
I have wrapped errors, reduced cyclomatic complexity, removed initial capitals from error messages, removed unnecessary conversions and else clauses. I have fumped my files ( gofumpt -w state/oci/objectstorage/objectstorage.go). I have removed most of the issues reported by the linter – except for some of the exhaustivestruct alerts; I am not sure resolving these will need to code that is better readable. Perhaps something to take up with the Dapr maintainers,.
6. Commit the new component’s sources with DCO
sign-off in the commit message
The Dapr instructions on making contributions stipulates that contributors sign-off that they certify that they wrote or otherwise have the right to submit the code they are contributing to the project by adding a Signed-off-by line to commit messages. This boils down to adding a Signed-off-by line to commit messages.
Signed-off-by: Random J Developer <random@developer.example.org>Pull requests can only be accepted if all commits contain this sign off: Each Pull Request is checked whether or not commits in a Pull Request do contain a valid Signed-off-by line. Note: Git even has a -s command line option to append this automatically to commit messages.
First I stage the changes I have made:
git add ./state/oci/objectstorage/
Then commit the changes onto the current branch with the sign off message:
git commit -s -m “Implement State Store for OCI ObjectStorage service”
7. Push changes to Remote (GitHub Repo)
The changes – new branch, two new directories and two new Go files – have been committed locally. In order to submit them to the Dapr project through a Pull Request, they need to exist in GitHub, in my forked repository. Therefore, I now need to push my changes.
Publish the new branch with the just committed changes to the remote repository on GitHub
git push -u origin oci-objectstorage-state
Checking in GitHub in my browser shows that the branch has arrived as have the two files in it.
My branch is already behind the master. Ideally a pull request is created from a branch that is fully up to date with the destination branch. So I will fetch upstream (get the latest commit from the Dapr main repository) and merge those changes into my branch (rebase my branch on the latest state)
Fortunately, fetching the upstream is easy – as expected – because no overlap exists between my changes and the recent commit on the master:
8. Create a pullrequest in the Dapr components-contrib repository from my
branch
Open either dapr/components-contrib or lucasjellema/components-contrib in the GitHub web portal. Click on Compare & pull request.
Fill in the PR form:
And submit.
9. Create an issue in the Dapr documentation repository
Go to https://github.com/dapr/docs/issues to submit a new issue in the Dapr documentation repository for adding documentation for using the new State Store component for OCI ObjectStorage, with a reference to the Pullrequest.
Click on New Issue,
Select New Content Needed and provide the issue details.
This Issue can be tracked through this link.
Update the Pull Request with the link to the Documentation Issue.
Next Steps
Now I have to wait for the Dapr maintainers to review my proposed change and provide feedback on how to finetune and improve – so as to be in line with the coding standards and best practices that I may not be aware of as newbie in Dapr and as novice in Go. Then I will also have to provide the documentation for this component, following instructions on new documentation contributions:.
I will update this article once this feedback arrives and let you know how my open source submission trip continues (or abruptly ends in case the feedback turns out to be devastating)
Resources
Contribution overview page on the dapr.io project site: – General guidance for contributing to any of the Dapr project repositories: https://docs.dapr.io/contributing/contributing-overview/
Developing New component page in the Dapr components-contrib repository: https://github.com/dapr/components-contrib/blob/master/docs/developing-component.md
(Issues on) Dapr Documentation Repository – https://github.com/dapr/docs/issues
Daprs docs with instructions on new documentation contributions:.
My issue of type Proposal in the Dapr components-contrib repository that describes the new State Store: https://github.com/dapr/components-contrib/issues/1400
The PR I created on the Dapr components-contrib : https://github.com/dapr/components-contrib/pull/1401
My issue in the Dapr documentation repo for documentation on the OCI ObjectStorage State Store
Much needed help on removing Git commits already pushed: https://stackoverflow.com/questions/22682870/git-undo-pushed-commits (I would not be so dumb as to push a commit with a private key in it, would I?)
Handy article on squashing multiple commits into a single one: https://www.internalpointers.com/post/squash-commits-into-one-git – handy if after pushing the commit, I find that go lint has some things to point about my code (and I do not want to burden the reviewers with multiple commits that make things look a little messy)
Also on Git: how to push a new local branch: https://stackoverflow.com/questions/2765421/how-do-i-push-a-new-local-branch-to-a-remote-git-repository-and-track-it-too
Using golangci-lint – https://golangci-lint.run/usage/quick-start
