I know how Web-APIs work, but how do APIs between two apps on one system work fundamentally?
If I write program A, that exposes an API X, and an Application B that calls on that API, how does that work from a compiler, OS and hardware standpoint?
In: 5
So OK, you’re talking very low-level. There’s not really one specific low-level answer. So, to oversimplify, I’ll pick one of many answers. This is going to address how
Let’s say our API is that one app has a way for the other app to make it display a message. That means there is a bit of code a programmer might refer to with this notation:
void DisplayMessage(string message)
This notation is based on a C-like syntax, which is one of the more popular styles for progamming syntax. Some languages call this “a function”, some call them “routines”, some call them “methods”. I’m going to use “method” because it’s what I’m used to.
From left-to-right what this is saying is:
> This is a method that does not return a value. It receives one parameter, a string named “message”.
The name isn’t super important to the program or compiler. It can be part of how it decides which bit of code to call but at the lowest level it’s all going to be numbers.
So again, in other words, this method needs an INPUT, the message it’s going to display.
The programs need to agree on what a “string” is. This has a lot of different answers. Again, I’m only going to focus on one.
One way “a string” can be represented is to use 1 byte for the length of the string, then let that many bytes come after. We also have to use an “encoding”, which is a set of rules for how we convert letters to and fron bytes. The UTF-8 encoding is very popular. (Technically it can use 2 and more bytes for some characters, but we’re going to ignore that for simplicity’s sake.) So the string “Hello” might get expressed as these 6 bytes, using decimal numbers for the bytes:
6, 72, 101, 108, 108, 111
That is, in order, “A length of 6, H, e, l, l. o”.
If both programs agree that’s how a string is expressed in bytes, now they know how to send strings back and forth.
Next they have to agree how they will communicate. In the old days, two programs could just reach out and poke each others’ memory. We don’t do that anymore. There are many ways two different programs can make a connection, but I am going to discuss a technique called “pipes”. These are a feature provided by the Operating System that lets the two programs *basically* have an internet connection with each other without the internet.
So first, both programs have to do some setup. When the programs start, each program tells the OS that it wants to use a pipe with a certain name. The OS sets aside some memory for that and gives both programs a special number called a “handle”. (Remember, nitpickers, I’m focusing on one implementation.) That “handle” is the ID for the “pipe”. If the program wants to say, “Send data to the pipe”, it has to include the “handle” so the OS knows *which* pipe.
If you want to get REALLY low-level, that means the program has to “call a method” that the OS provides. How’s that work? Well, the OS has what’s called a “calling convention”. That is a set of rules for how programs can handle this set of steps:
1. Set up the inputs for the method.
2. Tell the CPU to execute the method’s instructions.
3. Let the CPU return to the instruction AFTER the one that said to call the method.
4. Get any “outputs” the method generated.
One common calling convention is for the sender to use a data structure called ‘a stack’. This is named after the stacks of plates you tend to find at a buffet restaurant. You “push” data onto the stack by placing it at the “top”. You “pop” data off the stack by reading the “top” item then moving “top” to the next item. So one common calling convention goes like:
1. Push the current instruction’s address onto the stack.
2. Push every “input” value onto the stack.
3. Tell the CPU to “jump” to the function’s address.
1. The function “pops” its inputs from the stack.
2. The function does its work.
3. When it is done, if it has “outputs”, it pushes them onto the stack.
4. Pop the “caller”‘s address from the stack and tell the CPU to “return” to the next instruction.
4. Pop any expected outputs from the stack.
5. Use the outputs.
Finally, there has to be a “protocol” between the two programs. That just means they need to agree how some bytes mean “Call THIS method”. Let’s just go really simple and say the first byte is a number that means one of the methods, and the DisplayMessage() above is “method number 3”. That means to write the string “Hello” the “sender” will have to send thse bytes:
7, 6, 72, 101, 108, 108, 111
That is, “I’d like to call method number 7, here is the 6-character string it requires.” Remember, this isn’t the ONLY way two programs could communicate, but it is ONE way.
One last concept: interrupts. Sometimes the OS needs to tell a program, “Hey, stop what you’re doing because I need you to do this thing.” Some CPUs have a feature called “interrupts” to allow this. What happens is the OS is able to send a signal to the CPU. That causes the CPU to save a little bit of information about what it was doing then immediately go to a predetermined instruction in memory. The OS has set up code at that memory address to figure out what program is currently running and jump to *another* predetermined address inside that program.
The effect is the program might be way off in one neighborhood of the code doing something, then suddenly you’ll see it “jump” to that predetermined location. In this case, that location is the code for “do this when the pipe says it has received data”. It will run that code and when it “returns”, the process happens kind of backwards. The program runs an instruction that tells the CPU to read back the data it saved, then the CPU “jumps” to the instruction it was executing before it was “interrupted”.
Now we can sort of talk higher-level about what’s going on, and you can understand what’s happening at the lower levels.
The “sender” decides it wants to send “Hello” to the other program. It already has the handle to the pipe they share.
The “receiver” also has the handle to the pipe, and it has configured some code to be at the memory location the “data has been received” interrupt will jump to.
So the process goes something like this:
1. The “sender” builds the “message” it will write to the pipe: “Call the DisplayMessage method with this string.”
2. It uses a “normal” method call to a method the OS provides that says, “Write this data to the pipe with this handle.”
3. The OS stores the data in a place set aside for the pipe, then sends the “interrupt” signal to the CPU with a little extra data stowed somewhere to indicate which pipe changed.
4. The CPU saves what it’s doing and jumps to the OS’s interrupt handler.
5. The OS’s interrupt handler looks at the data step 3 saved and figures out the “listener” for this pipe is the “receiver” program.
6. It pushes the current instruction’s address onto the stack.
6. It finds the memory address of the “receiver”‘s interrupt code and tells the CPU to jump to it.
7. The “receiver”‘s code reads the message from the pipe’s memory.
8. The “receiver” checks the first byte and notices it needs to call DisplayMessage and will need to read a string.
9. The “receiver” checks the next byte and determines the string will be 6 bytes.
10. The “receiver” reads the next 6 bytes and stores them in memory.
12. The “receiver” pushes its current instruction address onto the stack.
11. The “receiver” pushes the string’s memory location onto the stack and jumps to the DisplayMessage method’s address.
12. “DisplayMessage” pops the string’s address from the stack then does whatever it does.
13. “DisplayMessage” executes a “return” instruction.
14. The CPU pops the stack and goes to that address. This is the address from step 13 representing code inside the interrupt handler.
15. The interrupt code is done so it executes a “return”.
16. An address is popped (step 6) and the CPU jumps back to the OS’s interrupt handler.
17. The OS’s interrupt handler is finished, so it executes some instruction to indicate that.
18. The CPU sets itself back up like it was before step 4 and continues doing what it was doing.
That’s a LOT. That’s why programmers don’t talk about the details much. Instead we like to agree on the details with each other and talk about it at a much higher level:
1. The sender writes a message indicating what it wants to do to the pipe.
2. The receiver reads the message and calls a method based on it.
At a high level, this isn’t different from how HTTP APIs work. There are just different parts in the middle.
Latest Answers