The Incompatible Mindsets of Software Development Communities
Why you should understand software development communities, not just your own.
I’ve recently been intrigued by the dynamics within and between software development communities, and particularly the mindset (mental models and attitudes) presented by each. If you find yourself needing to collaborate with a software development community different from your own, then understanding their culture is important.
This could mean working with a different team inside your own company, contributing code to an open source project, or perhaps selling a tool, library, or service for the community to use. You shouldn’t assume your approach to developing software will be looked upon favourably by other communities.
In this blog post, I’ll provide my definition of “Software Development Community”, explain why it’s important to understand communities (not just your own) and will describe four of the communities I’ve been exposed to in recent years.
What is a Software Development Community?
Back in the good-old-days (sometime before the 2000s) it was common to develop applications from scratch. You either wrote the source code yourself, or searched various FTP sites or Usenet groups for “free” software to incorporate. It was painful to construct a full application, causing you to implement the same code over and over again for each new project.
The 1990s saw the introduction of downloadable language runtimes and compilers, package repositories providing reusable code libraries, and frameworks with pre-architected skeletons for a new application. All of these worked together to make application development far more efficient than ever before.
To give examples, we now have:
The Java language, with the Maven package repository, and frameworks such as Spring and Akka.
The JavaScript language, with the NPM package repository, with backend runtime environments such as NodeJS with the Express framework, or front-end frameworks such as React or Angular, running in modern web browsers.
The Python language, with the PyPi package repository, using the pandas library to perform data science operations, or PyTorch for machine learning, or Django for web applications, or many other libraries for many other purposes (Python is very popular).
The Ruby language, using the RubyGems repository, and the Rails framework for developing web-based applications.
Using a range of languages, the Serviceful community creates applications using a collection of pre-built services, typically running on a cloud provider such as Amazon Web Services (AWS).
And the list goes on…
Each of these development environments has been successful enough that communities formed around them. Senior members of the community contributed content (libraries, documentation, training videos) attracting more consumers, resulting in an ever-growing community. Rather than building applications from scratch, new community members could build upon the frameworks and libraries that were already proven effective.
The Community Mindset
What’s interesting about communities is they share a common mindset, a set of attitudes or beliefs about how software should be developed. This is because communities shape their members toward a common set of values, but also those communities attract similar people in the first place.
For example, a common agreement in the Ruby-on-Rails community is that developing applications should be fun and easy. Therefore, the Rails framework and the corresponding libraries (known as Ruby Gems) are designed to support minimal configuration and therefore “just work”. As a result, it’s possible to construct very impressive Ruby-on-Rails applications with very little code, in a short amount of time.
In the JavaScript community, using NodeJS, all I/O calls are expected to be asynchronous in nature, not allowing the main application thread to block. Any library function using blocking code is frowned upon by the community as it negatively impacts system performance. In contrast, the Ruby-on-Rails community avoids this asynchronous approach, due to the excessive complexity of reasoning about program flow.
Finally, the Serviceful community is comfortable using Infrastructure as Code (such as AWS CloudFormation) to deploy and stitch together services, which is something the Ruby and Rails and NodeJS communities find unnecessarily low-level, therefore shying away from.
Despite their benefits, the downside of communities is that members are at risk of developing a single-minded approach, believing there’s one preferred way to solve a problem. A particular library will be recommended, a specific coding style must be used, or that “everybody does it this way”. This may be true within their community, using their agreed-upon set of values to make a decision, but it won’t always make sense across community lines.
I find it’s rare for developers to be deeply involved in more than one community. It can take years to become an expert in a programming language, a set of libraries, a collection of tools, or a framework, so there’s usually no time to stay actively involved with multiple communities. Not to mention, monitoring the community’s blog posts and YouTube videos requires significant effort. Focusing on one language or framework is therefore common.
Most developers have a community they feel very comfortable with, but will dabble lightly in other languages or frameworks when necessary to solve a problem. While branching out, it’ll take them longer to learn the language syntax, which tool to use, or the specific functionality of the framework they’re now working with. Overall development will be slower, but if motivated enough they’ll eventually finish the task.
Why You Should Care
Simply put, it’s easier to work alongside a community if you take time to understand their values, and the historical context around why they prefer certain approaches.
This can take many forms - you may be collaborating with an arms-length team within your own company, or need to dabble in a code base written in a different language, or perhaps you’re interviewing a job candidate from another community. You might even be a software vendor selling a tool or service into a new market.
Here are some differences to keep in mind:
Different communities use different words for the same concepts, so being an outsider can be confusing. That is, until you take five minutes to read the documentation, only to realize you’re familiar with the concept but by a different name! Of course, you must also start using those words when interacting with the other community. Is the correct word stream, or channel, or iterator, or …?
Along those same lines, the frameworks and libraries each have their specific set of APIs, which are similar to those of other frameworks. For example, every framework has a “read” function, but I can never remember the exact name of the function, or the exact parameters. I find my IDE’s auto-suggest functionality to be super helpful for prompting me, but bookmarking the online API documentation is also a must.
Every development community has an opinion about “speed of execution” versus “speed of writing code”, based on the application they’re building. A C/C++, or Rust programmer spends effort on efficient use of CPU and RAM, with the goal of faster execution. In contrast, Ruby-on-Rails developers want to minimize the time to develop a new feature, being less concerned about execution performance. Both approaches are perfectly valid, and depend on their end customer’s needs.
Similarly, the style of coding will differ between communities, depending on domain of the application being constructed. Systems-level programming (C/C++, Rust, or sometimes Java) can involve the implementation of complex algorithms using advanced computer science concepts designed to achieve performance and reliability. In contrast, a data scientist using Python, or a business application developer, using Ruby-on-Rails, will use a more domain-focused programming style to meet their customer’s needs.
Types or no types? This has been an ongoing argument for many years, with different communities preferring different styles. It can be faster to author code if you don’t define types for your variables and don’t need to worry about type mismatch errors. In contrast, many argue that type annotations make their program more solid and less likely to fail, which in the end speeds up development. With types being added to Python and TypeScript (on top of JavaScript), the communities are gradually converging.
Distributed system versus monoliths? It can be argued that distributed systems (multiple programs communicating via a network) are important when scaling your software. But, it’s equally argued that developing in a monolithic style (all code in a single program image) is far easier to comprehend, and easy to debug. Both groups are correct, but expecting the monolithic community to use a message queue or a distributed workflow service will be challenging.
Finally, each community has their preferred blogs and video channels where they share knowledge. Unfortunately, there’s a risk of becoming insular because recommendation engines are good at showing you similar content to what you’ve already viewed. It takes a deliberate effort to search for content produced by a different community.
To close off the discussion, let’s finish with a quick survey of some of the communities I’ve been exposed to in recent years. I won’t claim to be an expert in any one of these areas, but I’ve done plenty of dabbling, and have encountered more than my fair share of friction in doing so.
Example: The NodeJS Community
NodeJS is a server-side runtime environment for executing JavaScript code, working uniformly across Linux, MacOS, and Windows. Besides sharing a common language with web browsers, NodeJS provides additional functionality for file system access, network access, and many other traditional server-side features. Although general purpose in nature, NodeJS is commonly used to build light-weight web application servers, using frameworks such as Express.
NodeJS is the most popular framework, according to the 2023 Stack Overflow survey. It can be used for long-running web application backends, text-based command-line tools, or short-lived functions that execute in a Function-as-a-Service environment, such as AWS Lambda. Due to this flexibility, and the prominence of JavaScript as a web-development language, it’s no surprise NodeJS has a large community following.
NodeJS provides a single-thread programming model, which seems counter-intuitive for performance-critical applications. However, due to the Promise-based programming model, NodeJS does an effective job of time-slicing a large amount of work onto that single thread, keeping the CPU busy. This is important to the community members who care about performance.
A “Promise” is a chunk of code to be executed some time in the future, whenever the CPU is available to do the work. They are typically chained together in a sequence, and can be dependent on the completion of other Promises, or on the completion of asynchronous I/O. In contrast, many other programming frameworks block the thread while an synchronous I/O operation is in progress. For NodeJS, we instead schedule a Promise to be executed once the asynchronous I/O is complete, without blocking the overall thread of execution. Experience shows this is a high-performance and low-latency approach to performing work.
Like other communities, NodeJS users share pre-written software libraries, tools, or frameworks in a repository. The NPM Repository contains a large number of packages that are easily downloaded and imported into a user’s application. As you might expect, packages that gain a positive reputation within the community become more popular and are typically held to a higher standard of quality than less popular packages.
In addition to using NPM packages, NodeJS developers are comfortable writing large amounts of custom code. However, due to the somewhat unstructured history of JavaScript (at least until 2015), there’s a wide range of programming styles in effect. A newer object-oriented syntax (with class and method definitions) is available for fans of object-oriented programming, but it’s also possible to write code in a functional style (data is passed as arguments and returned as a result). Even though it’s harder to debug, use of global variables is still commonplace in a lot of software.
There’s no disputing that the NodeJS community is very strong, for a variety of reasons. Sharing the JavaScript language and package repository with frontend (web browser) code is a key advantage. From an efficiency point of view, the use of Promises and the light-weight runtime environment is a key benefit for faster development cycles. This community will likely be active for a long time.
Example: The Ruby on Rails Community
Ruby on Rails (2004) is the web application framework which made the Ruby programming language (1995) famous. You can create a complete web application in minutes by generating a templated application, then incrementally adding new features with just a few commands. The whole premise is that development should be fun and easy, allowing you to focus on your business logic rather than the underlying plumbing (which tends to be the same for every web application).
Rails follows a “convention over configuration” approach where a pre-defined set of standards (such as naming conventions) allows the parts of your application to work together seamlessly. You are welcome to go against these conventions, but doing so makes your coding effort more challenging and could break parts of the framework in unexpected ways. It’s therefore common wisdom to always follow the standards.
For example, if you create a User class as a subclass of the ApplicationRecord class, Rails creates an underlying SQL database table named users, with all the necessary columns. Querying that table is as simple as writing User.where(name: 'David') which invokes the relevant SQL command under the hood.
Overall, the Rails architecture is prescribed for you. Using the Model-View-Controller pattern, developers add their code into the correct classes (again, with naming conventions), allowing the framework to know exactly where to find the relevant code. Once a Rails developer has learned these conventions, navigating the code base is highly efficient (although it can be very challenging for outsiders!)
The Rails philosophy is to write as little code as possible. Developers prefer to solve problems by downloading Ruby Gems (packages), which ideally should just work without much configuration. A one-line solution to a problem (which invokes a Gem) is preferable to a 10-line custom-built solution. Writing a long Ruby method (> 10 lines) is often frowned upon.
The Ruby language is very dynamic, both with its variable type system and with how it allows dynamic interpretation of code. This flexibility allows very powerful programming constructs, such as DSLs (Domain Specific Languages) to be added into a Ruby-based application. On the downside, it’s nearly impossible to know that a Ruby program won’t have any syntax errors, until the code is actually executed. Unit testing has become a vital part of any Ruby on Rails application, to find errors at development time.
Overall, Ruby on Rails is one of the less common frameworks (see the 2023 Stack Overflow survey), but does have a devoted user base who enjoy the flexibility of the Ruby language and the conciseness of adding new functionality to an application. The community’s mindset is has therefore developed around these qualities.
Example: The Serviceful Community
The concept of Serviceful is relatively new, but is loosely-defined as being a programming model connecting together an array of existing services, often running in the cloud (such as AWS). A Serviceful program invokes service APIs, orchestrates the flow of data from one service to another, and handles error conditions that may arise. The community has developed around the tools and patterns used to build Serviceful applications.
There are many characteristics of Serviceful development, including:
Existing services provide functionality that we don’t need to implement ourselves. These services are easy to use when managed by a cloud provider.
Cloud-based services can scale in an elastic way. That is, you only pay for what you use, and can scale rapidly if demand for that service suddenly increases.
You’ll still need to write some custom code, but it’s often quite short. The paradigm of Event Driven Development allows small functions (such as using AWS Lambda) to execute in response to events taking place (a file is uploaded, an HTTP request is received, a timer expires).
Communication between services is done via creating and dispatching network messages (e.g. HTTP), rather than calling functions within the same (monolithic) program. This introduces the chance of partial failure if the remote service is unresponsive.
Existing application frameworks (such as NodeJS or Ruby and Rails) still have a place in this model, but are considered to just be one of the many services in the overall application (aka “microservices”). Communication with these services is usually done via the HTTP protocol.
Much of the time, the Serviceful community is referred to as the Serverless community. Although there are similarities, we shouldn’t exclude services that require knowledge of the underlying servers. Instead, a better comparison would be the Distributed Systems community, which focuses on communication between services over a computer network. The key observation is that many of the community’s challenges (distributed programming, network protocols, resilience) have existed for many decades.
In the Serviceful community, it’s normal to include Infrastructure as Code, such as CloudFormation or CDK, as part of the application. For example, creating and deploying code to AWS Lambda functions, provisioning databases, defining message queues, and specifying event routing rules are an important part of application development. Members of the Serviceful community must possess a strong understanding of cloud concepts, such as services, resources types, and permission rules.
In many companies, the Serviceful community is a partner to other communities, such as NodeJS or Ruby on Rails. That is, the Serviceful community are the members of the “Platform Team” who focus on the cloud infrastructure. Meanwhile, the NodeJS or Ruby on Rails community focus on the business applications that execute on the cloud-based servers, or within the Docker containers or Lambda functions. This separation of responsibility is quite common, especially in larger organizations.
Example: The Data Science Community
The final group I’ll mention is the Data Science community. These developers focus on the hidden meaning buried in large amounts of data. With the internet filling up with more raw data every second, there’s a lot to analyze!
Data Scientists have a statistics or mathematics background, rather than a computer science or engineering education. Their community is focused on languages (such as Python, R, or SQL) that support querying and summarization of data. Their code reads data into tables (rows and columns), performs analysis of the data, then outputs a summary of the key findings.
Common operations for data scientists include translating data from one format to another (such JSON to tabular format), cleaning the data by removing outliers and invalid data cells, querying tabular data to find trends, and joining data from different sources. Many of these operations are performed interactively using command line scripts, or Jupyter Notebooks that allow visual representation of the results.
Members of the Data Science community focus their skills on writing scripts to perform data transformation, often running the scripts on small datasets on their local machine (desktop or laptop). The community is large, and a variety of data manipulation packages have been written and shared widely. One of the most well-known Python libraries is pandas, providing a set of functionality for manipulating tabular data.
Given their statistics background, Data Scientists often rely on Data Engineers to put their code “in production”, using their computer science skill set. That is, instead of running their scripts on their local desktop, a Data Engineer has the skills to automate running the script on a large amounts of data, often using a compute cluster (such as Spark, or AWS Step Functions Distributed Map).
As you can see, the approach and mindset of a data scientist is quite different from those in the other communities we’ve discussed.
Conclusion
The takeaway message is that software development communities exist as a mechanism for sharing tools, libraries, frameworks, and documentation, to help build software applications. They form a common mindset by which the community members can make choices, all with the goal of building software more efficiently.
If we wish to interact with another community (other than our own), it’s important to understand the day to day life of a developer in that community, and the values by which they operate.