Consuming, Contributing, Collaborating: Open Source from all sides by Ron Minsky
19:392022
More people outside Jane Street now contribute to Dune than inside it
A trading firm with nearly 2,000 employees and no open source business model now contributes a large fraction of the improvements in each OCaml release and has published around 240 packages. Ron Minsky, speaking from his experience at Jane Street, shows why that effort pays off. For any company that only consumes open source, the open question is which way of engaging fits, and what it really costs.
See four real open source strategies and the three principles that keep them sustainable
Ron Minsky explains, through Jane Street's experience with Mercurial, OCaml, exported libraries and the Dune build system, how a company whose business is not open source can still benefit from contributing to it. He argues that sustainable open source work must align with business goals, that companies should choose their mode of engagement deliberately, and that tooling decisions shape the experience of internal and external contributors.
“Because I think an important thing to remember is contributing in open source is a repeated game, right?”
“Today, there are more people contributing to Dune outside of Jane Street than there are people inside of Jane Street. So I think of that as a pretty big success.”
What you will learn
Map the different ways a company can engage with open source, from funding consultants to launching community-first projects.
Justify sustained open source investment by tying it to concrete business goals such as ecosystem health, hiring and integration.
Decide when to keep an internal fork and when to upstream changes so you do not drift away from the wider community.
Design hosting and contribution tooling with the friction for both internal and external developers in mind.
Build trust with maintainers by treating contribution as a repeated game and proposing well-engineered, high-quality changes.
Details
Level
Intermediate
Format
Talk
Technical depth
Practitioner
For
Mid-career, Senior individual contributor, People manager
Length
19:39
Published
July 2022
Questions this talk answers
Why would a trading firm invest in open source when its business model is not open source?
Minsky says a healthy ecosystem gives the firm outside software to use, makes its main language attractive to recruits, and encourages external tools to integrate with its code.
When is hiring open source consultants not enough?
At Jane Street the consultants could not move fast enough because they were not inside and could not see the firm's problems, repo or workflows, so the firm started contributing directly while keeping the consultants to generalize and upstream the work.
How can a company publish code from a private monorepo?
Jane Street exports a chosen subset of its internal code to individual GitHub repos once or twice a week and imports issues and pull requests back into internal systems so developers can respond.
What made Dune succeed as a community project?
Jane Street hosted Dune on GitHub first and brought it internally periodically, giving outside contributors the same experience as anyone else; today more people outside the firm contribute than inside.
What are good practices for contributing to open source projects?
Minsky advises thinking about what is best for the community and treating contribution as a repeated game, investing in the engineering quality of changes so maintainers come to trust and accept them.
Open source contribution is a repeated game (69 s)
Speaker: Hi, everybody. So I am here today to talk about open source and in particular what it is like and how you should approach or how you can approach engaging in open source communities kind of from a kind of company and business perspective. And I just wanted to say folks should feel free to throw questions into the chat as we go. I'll be paying attention to that and try and pick off some questions later.
along the way to answer. So the whole talk is basically going to be from the perspective of my experience working at Jane Street. And so I want to spend a few minutes talking about who Jane Street is just kind of so you can. Whoops. Sorry, I lost my video there for a second. Talk about who Jane Street is just so you can get a sense of like where the talk is coming from the perspective that this all comes out of. So Jane Street is a trading firm and the basic business we have is that we are liquidity providers. We are people who are kind of middlemen. If you want to make an investment and you buy some security, then there's a good chance that we are on the other side of that purchase. If you decide to sell a security, there's a decent chance that we are on the other side there as well. And it's a kind of it's sort of like the moral equivalent of working in a grocery store. You know, you buy from one, you sell to the other, you try and make money from the gap between the price you buy and the price you sell. And that all kind of sounds, you know, pretty pedagogy.
But it's like a pedestrian. But it's actually really complicated. We have been doing this for 20 years. We have nearly 2000 employees. We operate in dozens of venues across many time zones and regulatory regimes, lots of different types of securities, tons of different strategies that we run. And the business is, as a result, enormously and intensely technological. We spend a lot of time and effort building software to simplify and scale, to optimize the delivery of data, to kind of make all of the things that we do more scalable and easier and make it easier for people to discover new ideas and build up new trades.
So that is not on the face of an open source business model, right? And so how does open source fit into that? Well, in some sense, open source fits into any serious technological enterprise. The foundations that we work on, the stones that we work on, the stones we stand on, are open source, right? From the operating system on up, we're using lots of open source software.
But that's just kind of the same as everybody else. In addition to that, we've spent a lot of time and effort over the years investing more deeply in various open source ecosystems, including supporting and engaging in pre-existing projects and creating and launching some of our own open source software. And that's really what I want to talk about, both how we've approached that and why we've done it. And I'll do that by walking through a few examples. So to start with, I want to talk about an example where we started out as a mere consumer of the software in question, and then moved on to kind of have a more deeply engaged role over time.
And this is with a piece of software called Mercurial. Mercurial is a version control system, a competitor to Git. It was actually designed at around the same time by almost the same people as Git. Both Git and Mercurial were built by Linux kernel developers. And Git is far more popular and pervasive than Mercurial. But Mercurial has an important niche, which is it's better suited to working with very large repos. And that's good for what we sometimes call a mono repo style workflow, where instead of, as you might do in a kind of common open source context, having lots of little repositories, one for each project that you have, you have one enormous repository that you put lots of different projects into.
And mono repos are useful, especially in large companies, because they simplify the versioning story. Instead of having to think, what version do I need of each project I want to include? Just every revision has a kind of complete and consistent decision as to what is all the software of all your dependencies. And a lot of big companies use mono repos, and a number of those, in fact, use Mercurial for some part of their version control to manage those mono repos, including Google, Meta, and Mozilla all use Mercurial materially.
So we started using Mercurial a long time ago, back in 2005. And as our use of it grew, I mean, we started, we just like downloaded it, it was software, we got it off the shelf, we used it, it was fine, it solved, it solved our problems. But we started doing more, growing the scope and scale of our efforts and writing more software.
And over time, we started hitting more stress points. And we didn't want to build out a big effort to work on Mercurial as a first class thing. So we tried to do something small, we hired some consultants. And this is kind of a common thing. Lots of open source projects have people who've taken on a role as consultants in that part of the ecosystem, who you can hire them to kind of help you out, help you with adoption, and also drive changes that are useful and necessary into the project itself.
And so we worked with those consultants both to advise us on how we approach things and also to kind of help build up more functionality in Mercurial as a whole to support the kind of uses we wanted. And that worked pretty well for a while. But eventually, as we continue to grow and continue to do more, we needed to do to move faster than the consultants could move on their own.
And the limitation for the consultants was mostly that they weren't inside. They didn't see our problems. They didn't see our repo. They weren't directly hooked into our workflows. And so we started getting more directly involved. And so we ended up rewriting significant bits of Mercurial and Rust. Most of Mercurial was implemented in Python. So for speed, we implemented some pieces in Rust. And we did it in a kind of narrow way that was fit to our needs.
But we kept the consultants on, and we worked with them to take that work, generalize it, improve it, and upstream it eventually. Because in the long run, we didn't want to have our own fork of the software. We wanted to be able to make the advances that we needed for our own use case, but also to kind of integrate nicely with open source.
So anyway, that's kind of one example of how we've engaged. Another case where we engaged more deeply, went from kind of starting out as kind of an engaged user and went on to become a primary stakeholder, is with OCaml. So OCaml is a programming language that's not used incredibly widely, but it's one that we're relatively well known for using.
We've used it for almost 20 years at this point. And we were pretty deeply embedded in the OCaml community from the beginning of our using of the language. But as with Mercurial, we started out as fairly naive users, at first just picking up the language and using it over time, submitting bug reports, submitting feature requests, and eventually starting to hire people to work on OCaml internally, to start submitting PRs and trying to get them accepted by the upstream people.
And that allowed us to kind of make a lot of important changes that made our internal lives better. But there was still a lot of friction there, because we went through the ordinary release process, and like, you know, it takes time to get your PR reviewed and accepted. And then there's the next stable release, which happens twice a year. And so it was a pretty long cycle.
And today, our engagement is much deeper. We have a really big team of compiler developers. We have about nine people working on that team. And we have our own fork of the language, where we're constantly making changes. Changes we make now land internally first. And then we iterate and refine and try and make those features better, quickly getting feedback from our internal users.
But we don't want to deviate away and break away from the larger OCaml world. So when those features stabilize, we try and submit them upstream and try and convince people that it makes sense as an addition to the language in general. And today, if you look at any given OCaml release, a large fraction of the improvements, surely not all of them, but a lot of the work that's done there comes from people who work at Jane Street.
So we're now really one of the primary people involved in that ecosystem. So Mercurial and OCaml are both cases where there's a pre-existing software ecosystem, and we were kind of joining into it. But that's not all we've done. We've also spent time taking our internal code and exporting it for the outside world. So if you go on to OCaml.org and do a search for things that are authored by us, you'll see 240-ish packages up there that came from Jane Street.
And that's all in almost a million lines of code. And that code is not a bunch of independent projects that are like sourced on GitHub and are done kind of fully out in the open. What's really going on is we're taking the work that we do internally, taking a subset of that work, and exporting it to GitHub, where other people can see and use it.
And the reason we do that is we get a lot of benefits from working in our internal world, working with our internal monorepo. But it just doesn't make any sense to take that monorepo and export its contents outside. First of all, it's got a lot of stuff that's private stuff that we don't want to export. Also, it would be kind of useless and unhelpful to people to have all of that stuff there.
Instead, what we do is we have a process that takes our internal software as it's developed and periodically, typically once or twice a week, exports it onto GitHub. Just the subset that we want to export broken into a bunch of individual repos for individual projects. And we tie things together in the other direction as well. When a new issue is filed or a PR is submitted, those are imported and kicked into our internal systems so developers can see that something is happening and respond to it in a reasonable way.
So this is all a lot of work. The tooling to hook these things up is not trivial. It's not stuff that people just kind of like have out there kind of in the outside world. It's stuff you have to build for yourself. And you might ask, why do we bother doing it? Because given that it's a lot of work.
And there's a couple of reasons. One is, from our perspective, it's really important to grow the OCaml ecosystem. We get a lot of benefit from all the software and work that's done on the outside in OCaml that we want to be able to use internally. And also from a kind of recruiting and hiring point of view, people here largely work in OCaml.
And we want that to be like an exciting ecosystem and exciting set of code for people to dive into. And so the more vibrant and healthy the community is, the better off we are. And so part of what we did in releasing all the software is to try to, you know, put out more stuff that makes OCaml more useful and encourages more people to use it and grow that ecosystem.
The other somewhat less obvious motivation for it is that by open sourcing all of this stuff, we provide something for other people to integrate with. There's lots of people on the outside who build useful libraries and useful tools and, in fact, extend the compiler in useful ways. And the more of our software that's out there for people to compile against and integrate with, the better those things will be matched to our needs.
And so in some ways by open sourcing all this stuff, we align the software that we build internally more closely with the external ecosystem. And that has a lot of knock-on benefits as well. Okay. So that's a case of like taking stuff that we have and like exporting it. But it's a clear like first class, second class thing.
The first class development experience is inside of our walls. And there's this kind of second class get export, which is like pretty good, but limited. There are also cases where we've decided to really go out and try and build whole communities around pieces of software. And maybe the most interesting example of this is Dune. So Dune is the, these days is the preeminent build system for OCaml.
So if you have like a big OCaml program and you want to do all the compilation and linking and pre-processing steps that are required, you can use Dune for doing that. And it gives you a nice, flexible, efficient system for that. It was developed at Jane Street. But we made a decision relatively early on to organize it as a kind of community focused project, which in part meant picking the community's tools over our own tools.
So instead of hosting it internally and exporting it to GitHub, we host it on GitHub and bring things, bring it internally periodically, right? So it changes where the frictions are. And it means that if you want to work on Dune and you're in the outside world, you have basically the same experience for doing that work as anyone else does.
And I think this was really important for making Dune a place that was attractive for other people to come and contribute. And it's really worked. Today, there are more people contributing to Dune outside of Jane Street than there are people inside of Jane Street. So I think of that as a pretty big success. And I think it also helped in the adoption.
Like if you look at OCaml's package manager, you'll see a larger and larger fraction of the packages over time are built using Dune. And that has other benefits. When we want to import someone's project internally, the fact that it's built in Dune, which is a build system that we understand very well, makes it way easier for us to pull it in to our internal systems.
Okay, so those are like three different projects, or actually four different projects, four different stories of things that we've done, open source work that we've done. They're not the only ones. There's a bunch of other neat things that you could look at. One of my favorite examples is a project called Magic Trace, which you go to magictrace.org.
You can see some beautiful analysis of exactly what Hello World does when you run it in a kind of elaborate detail that highlights that Hello World is way more complicated than you might ever have imagined. And that's another example of a piece of software that we've built as a kind of open source first piece of software. But anyway, these are examples, but there's lots more stuff that we've done over the years to kind of engage with the open source world.
So those are some examples. What are the takeaways from all that? What are things to remember? So the first one that I think is really important is if you're going to do open source work in the context of a business, in the context of a larger organization, I think it's really important to align that work with the kind of needs and goals and requirements of the organization.
So I want to be clear. Personally, I love open source software. It's kind of what I cut my teeth on, how I learned to be a software engineer. And I get a lot of personal satisfaction from doing stuff that contributes in this kind of public way. And both in terms of like contributing to existing projects and open sourcing our own stuff.
I find it enormously satisfying. But building open source software and doing it well and sustainably is really hard. And, you know, in some sense, the excitement that you might have around open source can get you a certain distance. But I think it's hard to justify a large and sustained level of investment unless you really think about how that open source work can line up with the goals of the company.
And I think part of what I think is interesting about the story here is even if your business model isn't fundamentally about open source, it's still the case that open source and active contribution and collaboration with open source communities can really benefit the business in a kind of useful and tangible way. The other thing that I hope these examples highlight is that there's a bunch of different ways that you can engage, right?
You can fund people to work on open source projects. You can communicate on forums and submit bug reports and write documentation. You can write your own patches and you can create your own freestanding open source projects to contribute to the world. And you should think of which approach you want as part of the open source strategy you're picking.
There are a ton of different choices and which choice you pick matters. It affects how you set things up. It affects what you can achieve. And so you really want to be intentional when you're making that choice. And the other thing is that, that's maybe less obvious, is that the tooling that you use matters an enormous amount.
Like every time you make a choice about where you're going to host it and how are you going to manage it and how are you going to manage contributions and things like that, there's a bunch of tooling implications. And that affects the friction, the friction for your internal developers and the friction for external people who are going to participate.
And so again, when you're thinking about what's your strategy for how to engage in open source, stopping and taking a deep breath and thinking, what is the user experience, what's the developer experience going to be like for people who are participating? And what should we do on the tooling side to make that as good as possible for the approach that we're taking?
I think it's a really important kind of fundamental question you should be asking early on in the process. Okay, so that's basically what I wanted to talk about. We have a few minutes left, and so if folk have questions, I would be happy to answer them. I see a lot of hellos, but not much in the way of questions, I think.
So yeah, if anyone has stuff to say, I would be happy to answer. Okay, and while we're doing that here, I'm going to... Ah, never mind. I was going to pull up a different one of our projects called Magic Trace to show people, but I think I don't have a moment. I think I can't quite do that now with how I have things set up.
All right. Ah, here. What are some best practices for open source contribution? That's a good question. Like, I'm not sure I have like a concise set of best practices, but I can say something about mindset, which is, I feel like when you engage in with open source communities, I think it's important to really think hard about what is best for that community.
Because I think an important thing to remember is contributing in open source is a repeated game, right? You're going in, you're not just trying to get one patch in or change one thing. You want people over time to trust and respect the things you're doing so that like when you go in to make contributions, people are actually willing to accept them.
So I think, like one thing I found to be really important is to not just like throw stuff up there without thinking, but to really take effort to think hard about the engineering and the quality of the changes you propose. And do that in a way that, so that people on the side of the kind of open source project come to think of your contributions as high quality things that they would be happy to accept.
Okay, and another question from you is, in terms of materials I can point you to. So there's a few things. So we actually have a tech blog that covers a lot of our work and is more focused on some of the open source stuff. So I think if you go to blog.jamesstreet.com, you'll see a bunch of posts that cover all sorts of different aspects of the work we do, but a lot of focus on the kind of open source side of the work.
You can also see actually all the concrete stuff we have. There's another website opensource.jamesstreet.com that has links to some of the major projects. And then github.com/jamesstreet is just our kind of our organization on GitHub that has all that there. All right. Well, I think we're at the end of our time. Thank you very much.
Transcribed automatically; names checked against the speaker record. Select a time to jump there.
Article based on this talk
Why a company with no open source business model should still contribute to open source
Ron Minsky walks through four Jane Street projects to show how companies can move from consuming open source to shaping it, and what makes that sustainable.
In short
Ron Minsky argues that open source can deliver tangible value to a business even when that business has nothing to do with selling software. Drawing on Jane Street's experience with Mercurial, OCaml, exported libraries and the Dune build system, he shows several distinct ways to engage, and concludes that the work lasts only when it aligns with company goals, is chosen deliberately and is supported by tooling that keeps friction low for everyone involved.
Key takeaways
Map the different ways a company can engage with open source, from funding consultants to launching community-first projects.
Justify sustained open source investment by tying it to concrete business goals such as ecosystem health, hiring and integration.
Decide when to keep an internal fork and when to upstream changes so you do not drift away from the wider community.
Design hosting and contribution tooling with the friction for both internal and external developers in mind.
Build trust with maintainers by treating contribution as a repeated game and proposing well-engineered, high-quality changes.
A trading firm with a technology problem
Minsky opens by describing Jane Street as a liquidity provider, a middleman on the other side of many securities trades, and compares the basic model to a grocery store that buys low and sells high. The reality, he explains, is far more complex: as the speaker cited, the firm has operated for 20 years, has nearly 2,000 employees and trades in dozens of venues across many time zones and regulatory regimes. That makes the business intensely technological.
Open source underpins all of this, as it does for any serious technology organisation. But Jane Street has gone further, investing in existing projects and launching its own. Minsky frames the rest of the talk around that deeper engagement and why it makes sense.
Mercurial: from off-the-shelf user to contributor
The first story is Mercurial, a version control system that, as Minsky explains, is better suited than Git to very large repositories and therefore to monorepo workflows. Jane Street adopted it in 2005 simply by downloading it. As usage grew and stress points appeared, the firm first hired consultants from the Mercurial ecosystem to advise it and build needed features.
That worked for a while, until the firm needed to move faster. "And the limitation for the consultants was mostly that they weren't inside. They didn't see our problems." (5:33) Jane Street then rewrote performance-critical pieces of the mostly Python codebase in Rust, narrowly fitted to its needs, while keeping the consultants on to generalise and upstream that work. The reason was strategic: "Because in the long run, we didn't want to have our own fork of the software." (5:58)
OCaml: becoming a primary stakeholder
With OCaml, the language Jane Street has used for almost 20 years, the firm followed a similar arc: bug reports and feature requests, then hiring people to submit pull requests upstream. The friction was the long cycle of review and twice-yearly stable releases. Today, Minsky says, a compiler team of about nine people works on an internal fork where features land first and are refined with fast feedback from internal users.
The fork is not an exit. "But we don't want to deviate away and break away from the larger OCaml world." (7:48) Once features stabilise, the team proposes them upstream, and Minsky notes that a large fraction of the improvements in any given OCaml release now comes from Jane Street engineers.
Exporting internal code, and why it is worth the effort
Jane Street also publishes its own code. Minsky describes roughly 240 packages, almost a million lines of code, exported from the internal monorepo to individual GitHub repositories once or twice a week, with issues and pull requests flowing back into internal systems. Building that tooling is not trivial, so he explains the payoff: a healthier OCaml ecosystem the firm can draw on, a more exciting language for recruits, and outside libraries and tools that integrate more closely with Jane Street's code.
He is candid about the limits of this model. "The first class development experience is inside of our walls." (11:38) External users get a good but second-class experience.
Dune: putting the community first
Dune, now the leading build system for OCaml, took a different path. Jane Street developed it but chose to host it on GitHub and bring it inside periodically, picking the community's tools over its own so outside contributors would have the same experience as anyone else. The result speaks for itself: "Today, there are more people contributing to Dune outside of Jane Street than there are people inside of Jane Street. So I think of that as a pretty big success." (12:49) Broad adoption also helps the firm, because projects built with Dune are much easier to pull into its internal systems. Minsky also points to Magic Trace as another open-source-first project.
Three principles for sustainable engagement
Minsky closes with three lessons. First, align open source work with the needs and goals of the organisation. He loves open source personally, but warns: "But building open source software and doing it well and sustainably is really hard." (14:58) Enthusiasm alone will not sustain large investment. Second, recognise the range of options, from funding contributors and writing documentation to creating freestanding projects, and choose deliberately because the choice shapes what you can achieve. Third, take tooling seriously: "And the other thing is that, that's maybe less obvious, is that the tooling that you use matters an enormous amount." (16:07) Hosting and contribution choices set the friction for internal and external developers alike.
In the Q&A, asked about best practices, he offers a mindset rather than a checklist: think about what is best for the community and remember that "contributing in open source is a repeated game" (18:11), so careful, high-quality contributions build the trust that gets future changes accepted. He also points viewers to Jane Street's tech blog, its open source site and its GitHub organisation.
Watch the talk
Consuming, Contributing, Collaborating: Open Source from all sides by Ron Minsky, about 20 minutes: https://www.womentech.net/video/consuming-contributing-collaborating-open-source-all-sides-ron-minsky
About the speaker
Ron Minsky delivered this talk for the WomenTech Network, drawing on his experience working at Jane Street as he describes in the session. The session record does not include a biography, job title or company details.
Questions this talk answers
Why would a trading firm invest in open source when its business model is not open source?
When is hiring open source consultants not enough?
How can a company publish code from a private monorepo?
What made Dune succeed as a community project?
What are good practices for contributing to open source projects?
Drafted with AI from the talk transcript. Editor review pending.
Frameworks and ideas mentioned
Monorepo workflow
Resources mentioned
blog.janestreet.com (Jane Street tech blog)
opensource.janestreet.com
github.com/janestreet
magictrace.org
OCaml.org
Cite this talk
WomenTech Network. "Consuming, Contributing, Collaborating: Open Source from all sides by Ron Minsky." WomenTech Network, 2022. https://www.womentech.net/video/consuming-contributing-collaborating-open-source-all-sides-ron-minsky