How To Design and build scalable APIs and Why it Matters by Emma Donery

Automatic Summary

Designing and Building Scalable APIs: Why It Matters

Hello there! I am Emma Dory, a software engineer at Opal Creek Limited and founder of Acute Labs Inc, a data analytics-based company assisting farmers. In addition, I support the women tech network as an ambassador and mentor up-and-coming professionals in the tech industry. I enjoy sharing my knowledge and experience via technical blogs on DEV dot to and my various projects on GitHub.

In this article, we shall delve into the importance of designing and building scalable APIs, focusing on the concept of future-proofing these fundamental software components.

Thinking Big: The Scalability Question

If you're a startup founder or a developer, have you ever asked yourself, "What if my startup or application unexpectedly takes off?" The sudden surge in user numbers and data transmission can be overwhelming, particularly if your API (Application Programming Interface) isn't equipped to handle the unexpected growth.

This is where the concept of scalability becomes crucial. Scalability is the ability of an application to handle a growing number of users and workload without compromising on performance. It ensures seamless user experience regardless of changes in application demand. For API developers, the thought of future-proofing therefore becomes non-negotiable.

Understanding API Scalability

A scalable API must:

  1. Enable smooth transition between software products
  2. Have efficient design and code optimization
  3. Possess a well-documented description

A robust API with scalability in mind can accommodate future growth, ensuring that your client's experience isn't affected when your application becomes the next big thing.

Elements of a Scalable API

When I categorize the elements critical to a scalable API, I group them into two: those specific to API and those adjacent to it. The specific ones entail a good API design and a comprehensive, user-friendly description. A well-structured API takes into account all possible future use cases and features good documentation for ease of reference and problem-solving.

Key Design Considerations for a Scalable API

A comprehensive approach to scalable API design would include:

  • Platform Independence: APIs must be flexible to any changes in the infrastructure or organizational operations, always ensuring optimal customer experience.
  • Predictability: Ensure that your API has consistent principles and follow basic conventions for request paths.
  • Interoperability: Your API should be adaptable to operate across different devices smoothly without hitches.
  • Adaptability: API development must be future-focused, adaptable to inevitable changes and updates.

Choosing to utilize widely accepted standards, keeping responses accessible, securing your API, and maintaining an up-to-date and accessible documentation are other salient and advisable principles in creating a future-proof API.

The Mantra for API Scalability

To ensure that scalability is baked into your API from the onset, here's a surefire mantra: "Conduct repeatable designs, long live version one, anticipate success, and expect the unexpected".

Framed to keep developers on their toes, this mantra magnifies the importance of being prepared for change, adaptability, and future success. Remember, in the digital landscape, your API's scalability might be the determining factor between an application's success or failure. Make sure you're on the victorious side!


Video Transcription

OK. So I'm going to talk about uh how to design and build scalable pas and why it matters. But I'll start by introducing myself um about me. Uh My name is Emma Dory. I am a software engineer at Opal Creek uh Limited.I am also a founder of Acute Labs Inc, it's a data analytics based company that um deals with uh helping farmers realize when it's time when it's the appropriate time to plant and um the re the the resources they need to use it ju just physically, things like that. Um Apart from that, I am an ambassador of this community women tech network and also recently I joined as an ambassador of women tech makers, Google and I'm so excited to be here. Um Apart from all of that uh during my free time, I can say, I mean girls is starting up in tech, those who are in their early career journey. So I have a mentorship club that are busy most mostly I do it online via social, social channels, not like I have a website or something. It's just slack channels, whatsapp. If any anyone is interested in joining the mentorship club. Um You can just talk to me via my socials uh on Twitter, on linkedin on Instagram. You just find me uh by the name Emma Dry. Uh I also write technical blogs during my free time.

I I blog on DEV dot to because I, I believe in writing, writing down what I know to help other developers learn what I'm learning or, or what I've learned because I also use a lot of articles when, during my learning journey. Yeah. And on github, if you want to check my projects in work, it's Emma cos 254 I guess that's the only, that's the only one that is, that is the only one that is different from others, all, all others. It's Emma Dony. Yeah. And so to our, to our topic, um I'll, I, I'll start by uh uh asking this question. Uh I want us to just think big. Um What if, what if you are a start up founder or you're working at a start up or? Um Yeah. II, I understand like a lot of companies are start ups. So what if, what if your start up takes off? We think about this big picture. Like, I mean, what if you start, you start a foundation today, you come up with an idea and you're like, um let me start up um an application maybe. Let me give an example with an application. I know you, you, you, you, you aren't even anticipating anything. Definitely because maybe you, you're thinking it will take time to grow, you know, and then if now you implement this idea and um out of the blues, people just love this idea and you, you, you start up, just kill up big enough like it just booms up without, without your knowledge, maybe you had anticipated like uh in 10 years it will boom.

But now it's, it comes up with a shock just one month and everyone just wants to use your application. Just it is that I thought, yeah, I know this is where um I'm, I'm I'm bringing this concept in terms of uh an API and um I want us to start by um understanding what is an API um an application, an A P A. It's just an acronym that stands for application programming interface. And now what is this application programming interface of? OK. Basically, I can define it uh as a set of rules that describe how computers and um app applications interact with each other. Or basically, it's, it's what the couples the fronted and from the back end. Yeah, simply that and it, it, it, it just enables that a transition between two software products. Now thinking of what I was asking if we think big, if we think about the future, I know you, you are an API provider or you use an uh A P A or you've created an EPA No, the, the, the big question is, is your A P A future proof? Like let's say you scale, let's say right now your, your EP is being used by 20 users or um 30 maybe 50 maybe 100. And now um you do a lot of, let's say online marketing or digital marketing or something and boom, your, your application is all over.

It's interested, it's interest, it has interested a lot of people, it has caught the attention or the eye of a lot of people. Now everyone is using an API is it future proof? Can it accommodate all this sudden change? Now, this is where the concept of scalability comes in. And um when you talk of scalability, OK. What is scalability when I when I see like let's build, let's design an A scalable API. So what is this scalability you're talking about um scalability? Uh It's just an the ability of an application to handle a growing number of users and load without compromising on performance and causing disruptions to use experience. OK. In terms of organizations now, uh can you can you uh when you talk scalability in terms of organization, can your organization accommodate change? Can it, can it be flexible enough like uh to accommodate maybe sudden change or future change or any growth? That's, that's what skill scalability and up when it when it comes to api I believe scalability should just occur at a very basic level. And this this basic level is about um designing uh you don't just start up like uh yeah, I want to create an EP I to link my front to my back end and then you create no, I mean, scalability is is it is a requirement and it is especially in this current technological era where um your application can really rush from relatively being unknown and in a matter of hours or in a matter of weeks, it is known and it is widely used, everyone has embraced it.

Like, I mean nowadays as as developers, when we develop things, you don't know, you just, you just don't, I mean, we we we we aren't such a anticipating for, for its success, but now no one knows the future, no one knows what the future holds. Um Let me give an example of an application such as um what is it called Snapchat? Yeah, Snapchat when Snapchat was created like um it was embraced so fast. So my, my question remains, what if, what if now is the developers who are developing a APAS that are associated with Snapchat? Now, what if they didn't have this this idea of scalability in mind? How how will the application now handle all this growth without now affecting customer experience or his experience? Because for every business, for any organization, for any application, I guess the important hazard is a client. We designed to, to make clients happy, to make their life easier.

Yeah, so now we have we have to have this, this idea of scalability. I understand like uh scalability right now might be a buzzword. And but for me, it is incredibly so essential, it is incredibly important to, to build robust API S because A P A providers must modify their interfaces honestly. They they must as, as applications grow exponentially and more larger and more complicated. Yeah. And why is this, why, why, why, why, why must they modify their interfaces to to accommodate scalability? For me? The answer is simple. It's it's to fulfill the growing need for his eccentric products and his experience. Yeah, just just that you know for, for scalability um if we have to employ scalability to to appear, we have to look at the elements or that are associated to scalability. Um for A P A, there are I I have categorized these elements into two. There are, there are elements that are specific to API and there are elements that are adjacent to it. A those are adjacent. OK. Aren't much of a concern in, in this, in this section because I believe such as infrastructure code quality and code optimization. Yeah. When you're writing an EPA code, you must write efficient code following the rights of their standards, etctc. Now these elements of scalability that are so specific to API one is API design, a good, a good A P A design.

It will enable you to scale both in complexity and in consumption like, sorry, sorry about that. Mm When designing, when, when designing an API and um OK, before now you start the process of building API if you design a good, a good API, I believe that. I mean um even it, it accommodates the future, you know, you, you, you're not designing the uh the API for for just now just basic, simple design and then you call it, you build it, you deploy it and we call it a day. No, we just structure the API. You think of all the use cases, you think of all the use cases that might happen in the future. Now you design with these use cases in mind. What if this happens? What if this other happens? What if this other happens? Such things you put all use cases into consideration while it is in and, and, and another element that is so specific to API is its description like I believe AAA good API and A SCALABLE API must be well documented. Don't just go use PDF S and et ce TC. Don't authenticate your documentations. Um A good documentation must be straight to the point one that our customer can easily relate to. And if maybe I have as a customer, I'm I'm having a an issue with an API, then I can easily go and find it, find it on the documentation or I can follow a documentation and probably understand whatever is happening.

And a pat for EPA to you just plan for your t it's just that simple. Um As, as I, as I said, uh a design first approach while designing and building scalable pis is so essential when building an ep that other people depend on availability and reliability are the key issues because uh for example, as a novice user and you're just looking for, let me say a yoga application or maybe a website that, that you want to, it to fulfill a certain need.

And then when you click a certain functionality, it like loads forever. You, you like have to wait for it to load, to load even high as a software engineer. I, I mean, I use other people's products as well and I don't believe I myself, I can just stay an application. Maybe I have, I have requested some information and I'm, I'm waiting for feedback and then the, the, I mean, like the stuff is taking forever to, to send the my request. I I I'll just go for an alternative option. Maybe I'll install the app, maybe even give a bad review and uh just go for, for some other efficient mm efficient application because I believe the problem with such an application, maybe there is a lot of load and the IP A is taking time now to, to handle all these user requests.

When, when, when designing on an A P A, you have to, to, to have skill in mind like you want to ensure that TAAP never goes down and it does it fall efficiently. It's fast for his users for, for users. Like if you request data, you want this data to be delivered as fast as you can because you want to maneuver, do what you want, accomplish the task that you want and move forward. Yeah. And um it appears that have beauty that's killing men. They honestly suffer from poor usability. It's it's so hard to use an EPA that he's so congested. Let me use the name congested. It's so I mean, I can even say confused, I can call it a confused P because if, if it can't be, then it is, it is losing its meaning. And if it has limited availability, I mean, the point of me even requesting for that is I expected the data to be available to me as fast as it can. And definitely such a pas also have are so open to secure security issues and you know, like between between clans to survive the A, the A P A that is requesting data.

I mean this data comes with even sensitive information and I believe it is should be secure enough to, to malicious users. Like people from outside cannot now come and collect other people's data and use it for criminal, criminal activities. So for, for, for me designing an A uh an API just sit down structure your design before you build it. And for, for some design considerations, there is platform independence, like don't let your api be dependent on our platform, it should be just understandable and consistent.

I mean, uh if, if things change, if anything changes in the future, if the infrastructure of the organization changes, it should also be able to change, able to be flexible enough to maintain the change while maintaining the functionality. Because the best goal is here is the best goal.

Here is customer experience. There should be optimal customer experience at all costs, whether it fails or it doesn't, there should be customer, customer optimal customer experience. Another design consideration we can have in mind is predictability. We have consistent principles is it's following basic conventions for request parts such as use resource names. Uh For example, if you're creating an API slash create product, a credit product API don't just rate API create products. No, you have a consistent convention for interoperability.

It's your API should have skill. You mean like if you, you, you, you're developing your API or you're using API for, for single clients, I mean operability might not be an issue. But now if you have, let's say that many devices, I can use it on an IOT device, I can use it on a on a mobile application, on a website, on a desktop. It it has to, to be this interoperable. It can, it can operate in different devices very nicely and very well for a vulnerability, it has not to, it's, it's, it just has to be adaptable to change future changes. Yeah. Uh basic principles of A P A design you, you build API is using commonly and widely accepted standards. OK? Such as you keep you. That's what, what I was seeing on the last slide following the busy conventions for request pads. A TCTC keeping your responses consumable, securing your API using documentation. Yeah, and securing your HI P A. You have to have limits. You can even use SSL or CS L. I mean TSL is just the success of SSL. But I see also many application using SSL right now and having a support is good documentation, making your A P A document, documents publicly accessible. You just, just another principle that's um gives no praise to a good A P A design.

Um I'm sorry if I'm, I'm talking too fast. It's because I'm seeing my time is up. Uh Yeah. And then II I have just uh constructed for mantra for scalability. If there is a repeatable design, you come up with a design that is, is repeatable. I mean, I believe in long live version one because you have to have a relaunch backup plan in case anything goes off the rails and then anticipate success, whatever I'm seeing like nowadays in this technological era, everything is changing. You don't know when to answer when, when your application or when your organization is going to scale, it's going to gain fame. So just, just be anticipating this success, even as Oscar Oscar WW said to expect the unexpected shows are that are more internet. You just have to expect the unexpected anytime anything can happen, your application gets killed and now you have to, to be prepared for scalability.