Jan Fornoff

Consistent Hashing

Sun, 14 Jul 2019 10:00:00 +0200

The Problem

You need to handle more requests than a single server can deal with.

First thought: Well, we can have two servers, duh.
Second thought: How do we decide where each request should go?

There are different methods of distributing requests between servers. The simplest method that intuitively comes to mind is Round Robin, where you allocate requests to servers one-by-one, starting over at the beginning when everyone had a turn. Obviously, the distribution of requests is very even (note that this does not quite imply perfect utilization, since not every request takes equal resources and time to process).

Round Robin

This doesn’t always work great. Imagine a case where you have to do slow computations to serve requests. You could cache the results to save redundant work and make response times faster. In this case, applying Round Robin would lead to redundant work and inefficient memory usage (due to each server computing and caching results).

Now imagine load increasing to a point where the servers must expire cache entries due to limited memory space. These expired cache entires will have to get recomputed or fetched, again doing avoidable work.

Here’s where request hashing can be useful.

Request hashing

A hash function (in the context of distribution) is not much more than a mathematical formula that projects a large space of values into a smaller one. This method lends itself well to mapping requests to servers (or “items” to “buckets”, which is essentially equivalent). This mapping is also stable, i.e., two identical requests are going to have the same hash value, thus mapping to the same server.

Load balancing can exploit this property to choose a hash function such that requests requiring the same cache data hit the same servers. When more similar requests hit the same servers, we increase the likelihood of cache hits! 🎉

For an example, let’s pretend that we have a service that could profit from caching, and we have three servers.

We could then use a hash to distribute requests naively as follows:

ServerToSelect(Request, NumServers) = Hash(Request) mod NumServers

Let’s also say we have three requests hashing as follows:

Hash(R1) = 3
Hash(R2) = 8
Hash(R3) = 19

This would give us:

ServerToSelect(R1, 3) = 0 // first server
ServerToSelect(R2, 3) = 2 // third server
ServerToSelect(R3, 3) = 1 // second server

Now let’s say we look at our monitoring and notice that an extra server should be added.

Ohhh no, elasticity!

Let’s see how the additional server affects the request allocation.

ServerToSelect(R1, 4) = 3 // fourth server
ServerToSelect(R2, 4) = 0 // first server
ServerToSelect(R3, 4) = 3 // fourth server

As we can see, after adding a node, the request allocation totally changed. This is problematic since all the cached data on the servers is now either useless or at least less relevant to the requests hitting it.

Can we do better than this? Well of course we can!

Consistent hashing

Consistent hashing provides us a more stable way of dividing requests in the face of changing server pools (original paper for those interested: 1).

Here is how it generally works:

We hash an identifier of each server in our server pool (e.g., their IP).
```
Hash(S1.IP) = X
Hash(S2.IP) = Y
Hash(S3.IP) = Z
```
We divide the Hash space between all servers (we’ll see how that works in the next section).
We hash requests, the server who “owns” that part of the hash space fulfills it.

The Visualization

Don’t freak out yet. This is a “Hash Ring” and acts as a visual way of reasoning about how requests are going to map to servers. We are going to assume that our Hash function returns a 64-bit integer. The ring forms by “wrapping around” from the maximum value (2^64 - 1) to the minimum value (0).

To determine the responsible server for a request, we look at the hash value for said request. Then we determine the “closest” server on the hash ring.

Implementation may work in different ways:

Start from the Request hash value, go clockwise until encountering a Server hash value
Compare absolute distances to the two closest Server hash values (in either direction)

For example: When looking at R3, absolute distance would map it to Server 3, clockwise rotation would map it to Server 2.

The important thing to realize is that every server “owns” a part of the hash ring, i.e., a range of request hash values that will map to it.

Upon adding a server, it takes over a part of the hash ring from neighboring servers. Upon removing a server, neighboring servers are taking over the part of the removed server. The most important part: most of the hash ring (and all request mapping therein) remains unaffected.

This unlocks the benefit that servers can come and go without large impacts on request distribution. Furthermore, this distribution technique handles limited views of the server pool well. If a server is unknown to some peers, e.g., when it has just joined the server pool, some requests will still hit the server that “owned” that section of the hash ring before. However, there is a high probability that it will still have warm caches for these types of requests!

Conclusion

Consistent hashing allows for more robust reassignment semantics when spreading requests across a dynamic set of servers. While slightly more complicated to implement, it’s still fairly simple.

Closing off, I want to note two more things. Firstly, dividing the hash ring space evenly between all servers is not inherently solved by consistent hashing. One approach that alleviates the problem is replicating servers to multiple points on the hash ring, this comes with the drawback of multiplying memory usage.

Lastly, the problem considered above is web caching. The distribution provided by consistent hashing also proves useful for distributed storage systems (i.e., storage systems where you cannot fit the data set onto a single machine). The division of a dataset between computers is then called sharding. Adding and removing shards benefits from consistent hashing as well, since it minimizes the data that needs to transferred between shards to keep the data base consistent. (Interesting paper: 2)

But this is a topic for another post. Bye!

On Circuit Breakers

Thu, 31 Jan 2019 06:48:45 +0100

Hit me baby, one more time

Circuit breakers are a software development pattern that aims to increase the reliability of a service in the face of unreliable external dependencies.

If your system does not function when it has no access an external dependency, its reliability is necessarily the minimum reliability out of all its dependencies, or likely worse.

Circuit breakers rely on a property of failing requests: If there is a bunch of failing requests, subsequent requests are likely to fail as well. There are many reasons why that might be:

The opposite service is down, crashed or overloaded
Network connectivity is impaired
Requests are rejected by the opposite site due to bugs or misconfigurations

All these conditions have in common that they probably won’t be gone after a few milliseconds. This impression is reinforced when we have tried a few times already and there is no recovery to be seen.

Why the name?

As you likely know, a Circuit Breaker is an appliance used in electrical circuits to cut the connectivity in case of harmful conditions of very high current (as it often implies a short circuit). It is providing fault tolerance for electrical systems. The Circuit Breaker pattern provides fault tolerance for software systems. So how does this concept transfer?

As we discussed above, if there is a bunch of errors when trying to interact with an external service, there is a point (e.g., after requests to the service failed consistently for some amount of time) at which we can assume that it is not terribly useful to even make the call anymore. The Circuit Breaker is a software component that detects when this is the case and “pulls the plug”. After that, it does not call the external service anymore. It just returns the error right away. Or even better, there is a fallback strategy in place.

Meditating on failure remediation

The interesting part about circuit breaking is in my opinion, that you have to think about your failure recovery. What can you do if this or that external service fails to respond to requests? Let’s think about this on a little case study.

On our shop homepage, we want to present some recommendations from products that the user might be interested in. The front-facing application queries a recommendations system. This system uses previous orders made to figure out suitable recommendations.

Someone tripped over the power cord of the purchase system, we can’t retrieve the previous orders anymore D:

How do we deal with this failure mode? We could throw our hands in the air in panic and stop serving requests altogether. Doesn’t sound great to you? Yeah no, it’s probably not. Right now is the time where we put on our thinking hats and deliberate on “What do we do if we can’t compute recommendations based on orders?”

We could:

Find a recommendation that we can always make (most popular products, anyone?)
Return an error to the frontend servers and tell them deal to with it. That’s a perfectly valid strategy, maybe it’s not even a big deal to not present recommendations on the home page.

Both strategies can be applied using a Circuit Breaker. The pattern is agnostic of how you deal with the detected failure. It allows you to not overload the external system further. Furthermore, you can respond quicker instead of running into timeouts over and over, when it is highly probable that it will happen again (this is known as the Fail Fast principle). This way, propagation of failure and slowness through the dependency chain is mitigated. Your frontend servers will go back to responding quickly after the recommendation system’s Circuit Breaker kicks in after a few query timeouts.

Conclusion

Circuit Breakers are a very useful pattern to deal with bursts of failure, preserving quick response times and facilitating a consciously chosen fallback strategy (whatever it might look like). There are a lot of examples from the practice where the same principle is applied (e.g., NGINX upstreams and Kubernetes services), but that is for another time. Until next time!

Introduction to the Transmission Control Protocol

Mon, 10 Sep 2018 19:53:51 +0000

This post continues the series on the fundamental protocols of the Internet, with a focus on the abstractions they provide and how they build on each other.

TCP (the Transmission Control Protocol) builds on top of IP (find the accompanying post here. Based on the best-effort delivery guarantees of IP, it adds reliable byte stream guarantees.

In plain english this means:

Messages sent are eventually received (Reliability)
Incidental reordering of arriving data fragments is resolved

TCP acknowledges the concept of inter-process communication (IP doesn’t have that, it just cares about how to get messages from one host to another). It is fundamentally based on connecting a “pair of processes” (if you are interested, RFC 793).

Staying with the theme of providing services to upper layers, TCP offers:

Ensuring that the receiving process can keep up with the senders pace (Flow control)
Utilizing the underlying bandwidth without overloading the network (Congestion control)
Ordering (using ascending sequence numbers attached to each data segment)
Reliability by data retransmission

These properties are what make TCP so popular, few people really want to think deeply about what happens when data segments are lost or reordered on the path over the network.

Bird’s eye view

For the rest of this post, I’d like to give you an impression how TCP is actually used by applications or upper-level protocols.

To start communicating with another process, it is necessary to establish a TCP connection. The end product of this is a connected pair of Sockets, essentially an abstraction allowing to read and write data through the connection (you can check out my post on Linux Sockets if you want to know the details). Symmetrically to the setup, there is a teardown sequence that is performed when you close a Socket on either side.

Buffering

The TCP implementations on both sides of the connections are keeping buffers for both transmission directions – receiving and sending. These are limiting the amount of data that can be:

sent without the other side consuming it
received without being consumed

Applications send and receive data using system calls (i.e., your application walks to the Linux kernel and asks it to send or recv from the network). In between recv calls, the Kernel buffers a finite amount of data in the Socket receive buffer. If that buffer runs full, TCP’s flow control mechanism allows indicating to the sending application to stop sending for now. The technical term for this sort of behavior is back pressure. The interesting part about this is that the sending application doesn’t need to know about this behavior, the send system call just starts blocking until the data can be sent. recv behaves similarly on an empty receive buffer – it just blocks until it has some data to deliver.

Conclusion

Summing up, TCP abstracts away the complications of dealing with an unreliable underlying network. Using these provided services, applications and upper layers can just focus on sending and receiving data over a reliable channel.

Introduction to the Internet Protocol

Sun, 27 May 2018 19:05:48 +0000

The Internet Protocol (IP) is a crucial underpinning of the Internet as we take it for granted nowadays. To be completely honest with you, there is two IP protocols (there is the “old version”, IPv4 and the “new version”, IPv6). Both are standardised in the form of RFCs (IPv4 and IPv6). There are many more RFCs specifying details of the technologies, augmenting the base RFCs.

However, this post is not about technical details.

What this post is not about

Differences between IPv4 and IPv6 (Spoiler: v6 is pretty damn awesome.)
How protocol headers look like
(It’s interesting but you can do it yourself, it’s just information in a predefined format.)

What this post IS about

Instead, this post goes over the Internet Protocol on a very high and abstract level:
How IP fits into the Internet protocol stack, the guarantees that it provides and how the OSI layers separate concerns.

Service: Giving and taking

IP resides at the Network Layer (Layer 3) of the OSI stack. The core idea of these layers is providing services to an upper layer, and using services provided by a lower layer.

Lower layers know nothing about how their services are used by upper layers.
Upper layers know nothing about how exactly lower layers operate.
It does not concern them either – it’s an agreed-upon contract of service delivery.

The service that IP provides is:

Addressing nodes on a network
Forwarding and routing packets between nodes on the network

In summary, IP provides upper layers with the service to send data to another node on the network without worrying about how to get the packet to the right place in the network topology. The upper layer (usually the Transport Layer, but there are exceptions) may be for instance TCP or UDP.

IP in turn uses services of the Data Link Layer (Layer 2), which provides delivery of data frames within e.g., a LAN (local-area network) or WAN (wide-area network). For the former, services on this level may be Ethernet or Wireless LAN.

As such, each layer has a well defined responsibility, i.e.:

Transport Layer knows about strategies of connecting pairs of processes (e.g., your web browser and a server) – it may provide e.g., persistent connections, reliability and flow control.

Network Layer knows how to navigate data from A to B within a network (the Transport Layer is completely oblivious of how to do that, it just assumes it can address someone and the packet will – most likely – make it there).

Data Link Layer knows how to transport the data between nodes in a network (e.g., moving data between your Laptop and Home Router using WiFi or Ethernet).

Guarantees

Now IP is a so-called best-effort protocol. Plainly stated, this means: “I can’t promise your packet will make it, but I’ll try, ok?”. Sending an IP datagram therefore does not guarantee that it arrives at its destination.

Sending may fail for a multitude of reasons:

Very busy Routers on the network path being forced to discard packages (also called “Congestion”)
The underlying physical medium causing the package to be delivered incompletely
Packages being intentionally discarded by e.g., Firewalls or Routers

IP does not attempt any correction in case this happens. This is decided by the upper layers (e.g., TCP can retransmit lost segments of its data stream when the delivery seems to have failed).

Conclusion

This post attempts a new style of writing that focuses more on the broad architectural strokes of what the Internet is made up of. There is a lot of material on technical details. However, I personally feel like intuition for the underlying thought framework is way harder to find.

Hope you found this useful, until next time.

Book Notes: "A Guide to the Good Life"

Mon, 19 Feb 2018 06:59:49 +0000

This post will outline the first three of the core lessons I took from “A Guide to the Good Life” by William Irvine.

Tranquility & Virtue

At the core of stoic philosophy stand the concepts of Tranquility and Virtue.

Tranquility is described as the “psychological state marked by the absence of negative emotions”, such as grief, anger or anxiety. However, this is not a zombie-esque state of “not feeling anything”, but instead the presence of positive emotions, untainted by negative ones.

Interestingly, Tranquility and Virtue are symbiotic. If you are not tranquil, your mind will be distracted by the negative emotions and your judgement will be less reasonable. This in turn prevents you from moving towards Virtue. Your emotions triumph over your intellect.

Adversities

Challenges and adversities in life are reframed into a sparring-like context. If fate hits you hard, you should regard it as a type of training situation.

Adversities we encounter in life are mere training. These things that we shudder and tremble at are for the good of the persons to whom they come.

We can observe how application of this thought changes emotional responses. Instead of being frustrated and resenting an unfortunate event (note how these are all negative emotions), you shall view the broken Nutella jar and lost Love as an opportunity to practice your Virtue.

Negative visualization

Contemplating bad things happening to you is powerful. Why?

Firstly, and less importantly, it helps you be mentally prepared if fate strikes. However, the more interesting effect of imagining yourself getting fucked over in every imaginable fashion, is that you start appreciating not being fucked in these ways.

Let me elaborate.

Usually, things go a little like this: You have quite a lot of stuff. If you’re reading this, I’m assuming that you at least have access to the internet. But you’re pretty much used to it at this point. And chances are, you want something right now, would just love to have it! Nothing would make you happier. Be it that slick new piece of tech gadget, that girl or boy next door looking so damn fine, a new pair of shoes. And I’ll be bold and state that after you get it, you’ll eventually lose your appreciation for it and it’ll stop giving you joy. This is natural, we adapt, even to what we have.

We are also dopamine junkies (the stuff in your brain that gives you that rush when you get something new). When our appreciation wears off and we grow accustomed to the thing we just wanted so much, we eventually want something new to give us a new rush. We start chasing the next thing.

Stoics think that we can rejuvenate our passion for the things we already have by imagining how it would be to lose them. This does not mean getting pessimistic or even fatalistic. This means realizing that you’re not entitled to have all these nice things.

All we have is on loan from Fortune, which can reclaim it at any time, and without advance notice.

This lets us re-learn to appreciate what we have. By that, we can cut the cycle of wanting and chasing the next thing in pursuit of satisfaction short. It puts our first world problems into perspective.

Again, observe how this lends itself to attaining Tranquility by taking away the negative emotion of being unfulfilled because you don’t have the new iPhone X, and replacing it with the gratitude for having a phone that allows you to talk to your loved ones from anywhere.

Choosing your battles

The core of the next lesson is categorizing things you do and face into three categories.

If you have complete control over something, engage with it fully. No one can keep you from succeding but yourself.

In contrast, if you have no control, your actions have no impact on the outcome at all. All effort invested is wasted. Therefore, you should not be concerned with it.

The category right between those, to me, is the most interesting.

Internal & external goals

Suppose you’re really ambitious in playing a sport. You play a match, try really hard and still, you lose. Pretty upsetting, right? Notice what your frame of mind is when you focus on the outcome of winning. This outcome is not fully within your control. Your training efforts and how focused you are improve your chances of winning, but you cannot be sure. Call this an external goal.

If you are really attached to being the winner, failing to win will be crushing. It will disrupt your Tranquility. Furthermore, if you’re behind on that match, you will probably get frustrated and play worse.

Now contrast this mindset with someone who plays with the goal of playing to the best of his / her ability and trying to get better. This goal is way farther within your control. Call this an internal goal. Even if you lose the game, you can attain your personal goal and retain your Tranquility. Additionally, playing your best will support your chances of eventually getting to win!

Internalizing your goals is advisable for matters that are not fully controllable by you. This isolates you from the negative emotions of uncontrollable outcomes. Meanwhile, you move forward.

Let’s wrap up for now, hope this was useful to you. I’ll see you on the other side.

Introduction to Linux sockets

Sat, 20 Jan 2018 11:14:58 +0000

This post will cover a little bit about the abstraction of a Socket in Linuxworld, what they exactly are and how they are generally used.

What are Sockets and how are they created?

Sockets in the most general sense are an abstraction of interprocess communication. They represent the endpoint of a communication channel (abstracting the details of the underlying transport).

To interact with Sockets, you use system calls. Those are commands that you send the Linux kernel who manages the communication and networking stack.

Sockets are created by using the socket() system call. Let’s deconstruct this call by looking at it’s arguments.

int socket(int domain, int type, int protocol)

Communication domain

The communication domain specifies a family of protocols that this Socket is going to live in. Most prominently, Sockets can be “scoped” to the local machine (Unix socket) or the internet.

UNIX domain (identified by the AF_UNIX constant)
IPv4 internet (AF_INET)
IPv6 internet (AF_INET6)
a bunch more, which can be found in the man page

Socket type

The Socket type specifies the communication semantics which are going to be in effect for this socket. Let’s go through the most common examples:

Stream sockets (`SOCK_STREAM`)

Stream sockets provide what is called a sequenced, reliable, two-way, connection-based byte stream. Deconstructing this in terms of guarantees provided: - Sequenced: Messages are received in the same order in which they were sent - Reliable: - No duplication: The same message is received at most once - No creation: Messages are not received unless sent (there are no random messages appearing out of thin air) - Validity: When a process sends a message to another one (given that they’re correct and not buggy), it will eventually be received

two-way: Both sides are able to send and receive
connection-based: A session is established prior to data transfer

The protocol in the Internet communication domain for this socket type is TCP.

Datagram sockets (`SOCK_DGRAM`)

A datagram is: > A self-contained, independent entity of data carrying sufficient information to be routed from the source to the destination computer without reliance on earlier exchanges between this source and destination computer and the transporting network. > RFC1594

Datagram sockets provide sending of connectionless, unreliable messages of a fixed maximum length. The protocol in the Internet communication domain for this socket type is UDP.

There are more types of sockets, which can be found in the man page

Socket protocol

In most cases, the int protocol is redundant, it is used when the communication domain has more than one protocol available for use (for example on SOCK_RAW type sockets). Otherwise it can be nulled and ignored.

Connecting Sockets

Obviously, Sockets are only useful when there is another program on the other end of the communication channel. So how do Sockets connect to each other?

Most commonly, this happens by a Socket connecting to another Socket that is listening. Let’s step through this by a TCP client-server example.

Server side

When a Server is starting up, it: 1. Instantiates a Socket through the socket() system call that we just saw. 2. Binds it to a name (in this case IP address and port) through bind() 3. Allows incoming connections on the Socket by calling listen() 4. The socket is now able to be connected to, essentially putting incoming connections into a queue 5. Calling accept() pulls connections from the queue and allocates a new Socket for them which is returned

Client side

When trying to establish a connection to a Server, the steps are: 1. Instantiating a Socket through socket(), just like in the Server 2. Calling connect() with the name / address that the Server socket was bound to (in SOCK_STREAM sockets this attempts to initialize the session with the Server) 3. Sending some data (e.g., the HTTP request) to the server by using send() or similarly write() (which behaves the same, but you cannot set flags) 4. Reading a response using recv() or read() - this call blocks (does not return) waiting unless the socket is set to use non-blocking I/O (by using fcntl()) 5. When done interacting with the Server, close() is called on the Socket, which deletes it and closes the established connection.

Summing up

Hope this has taught you a thing or two about Sockets and given you a bit of insight on how Sockets actually work close to the metal.

Until next time!

Documentation Driven Development - or why names matter

Sat, 09 Dec 2017 10:42:34 +0000

Today’s post will be a pitch for why you should be sensitive to how you refer to concepts in your code and documentation.

First statement: Advent of Code is fun as heck and you should do it.

While solving the Day 7 problem (My solution here - Spoiler alert) I got stuck. I knew what generally needed to happen, but couldn’t quite get it right.

Everyone gets stuck, and people deal differently with this, I usually start scribbling on paper. Today I wanted to try something different.

@doc """
<Your mental picture of the problem>
"""

Problem solving is verbal

Especially when writing code, you need to name the concepts and abstractions you’re working with. And if you’ve been writing source code for an extended period of time, you will probably agree with me when I say that there are parts of code that just “click” when you read them.

Yeah that’s obviously doing X.

Is it obvious though? Is it that the solved problem was just really simple that caused you to sweep over the code without any mental friction?

I’ll argue that it’s not. Instead, I’ll claim that the author of the code spent time figuring out what the fragments of the problem should be called to give you the right impression about what they’re there for and where they conceptually belong.

Moving from this point, it’s conclusive that you’ll probably have a better time understanding your own code as you write it when you pay attention to what you call things. Every time you have to step back from writing code and wonder Wait, is this right? What was I trying to do again?, this will yield dividends. You can re-read where you came from and pick back up with clarity.

Now the fitting name usually won’t pop into your head right away. Here’s where writing documentation (even before the code exists) can help you map out what’s actually going on with the problem.

Document to establish vocabulary

When you read through a Jira ticket or someone tells you about a problem that’s been bugging them, you subconsciously start mentally modelling the pieces of data you’re dealing with and already start sketching out what the rough solution steps would be.

If the problem has any appreciable degree of complexity, you will most likely not get it right by just eyeballing the solution approach.

This is where it helps to just write down what you conceptually want to do.

@doc """
This parses tree nodes with their child node names from the input string and
rearranges them into a traversable Tree data structure,
represented by the root node containing its child subtrees.
"""

def build_tree(input_string) do
end

From here, it is pretty straightforward to “Write the code you wish you had” (which is a saying often used in the TDD and BDD community, stating that you can dream up how ideal high-level code should look and not care about the underlying implementation yet)

def build_tree(input_string) do
  input_string
  |> parse_nodes()
  |> arrange_nodes_into_tree()
end

From here, you can drill down and start wondering “What do I need to do to parse this string into a list of nodes?”

Also note how the documentation-writing also eased us into establishing the core concepts of our solution model. This is the vocabulary of your solution. Do not deviate from it, decide!

Conclusion

Taking your hands off the keyboard for 10 minutes and figuring out in which terms you want to reason about the problem at hand is not unproductive.

If you feel like the text on your screen is gibberish, try to descriptively state what this code is trying to do, find out what’s clouding the clarity and rename something for the better.

Farewell!

Continuous deployment for your CV

Fri, 03 Nov 2017 18:18:37 +0000

The goal of today’s post is to get a little whiff of Continuous Deployment, especially build pipelines.

First off, I need you to realize that sending CVs by Email is not acceptable, because this is 2017. Are we on the same page? OK.

We’re going to put our CV into version control and build a PDF out of it as soon as we push a new commit.

Frequently Asked Questions

Is this necessary? No.
Are we doing it anyway? Yes.
Why? Firstly, because we can. Secondly, because it teaches us about automation.

The Plan

What are we specifically trying to achieve? - Being able to change our published CV by pushing a new commit to a Git repository - Having a stable URL at hand that always refers to the latest and greatest version of our CV

Secondary priorities: - Not having to host anything ourselves (not because we can’t, but because it’s not necessary and less effort)

Automatic building of PDF

Our goal is to push a Git commit and somehow get a PDF out of it.

To keep it super simple, we are looking for a Git repository provider offering build pipelines so we don’t have to wire up Hooks to trigger build jobs. I considered Bitbucket and Gitlab. Eventually chose Gitlab since Bitbucket severely limits the amount of available build time and makes public downloads harder.

Build pipelines are declarations of what needs to happen to build something. Usually they run automatically when a new push a Git repository happened. Our Gitlab Pipeline will do exactly that for us, build us a PDF whenever we push.

Here is a Gitlab pipeline definition.

stages:
  - build

compile-cv:
  stage: build
  image: blang/latex
  script:
    - bash build-cv.sh
  artifacts:
    paths:
      - "build/CV.pdf"

Quick walkthrough: 1. stages just allows grouping your pipeline into stages - what a shocker. That’s pretty cool for running things that don’t depend on each other in parallel.

compile-cv is a “step” in the build that is run in a Docker container
- based on the Docker image blang/latex, which provides a LaTeX build environment
- running a shell script that basically just executes pdflatex
- defining the pdflatex result - build/CV.pdf - as an artifact

Job artifacts in Gitlab pipelines are super neat, since they are downloadable, and allow us to link to the most recent artifact under a stable URL!

Making the PDF publically available

This pretty much just requires you to set the Gitlab repository visibility to public. I’m sure you’ll figure out how to do that if you really want to.

Eventually, you will be able to call an URL that looks a bit like this, and get a PDF download (Rockstar developer tip: Log out of Gitlab to see if it is actually publically available).

https://gitlab.com/jfornoff/cv/-/jobs/artifacts/master/raw/build/CV.pdf?job=compile-cv

Obviously your URL will vary, the general scheme is:

https://gitlab.com/<namespace>/<project>/-/jobs/artifacts/<ref>/raw/<path_to_file>?job=<job_name>

Getting a pretty URL

This URL is totally not memorable and you’ll need to look it up, we can do better.

I’ll give you two options that might work for you:

1. URL shortener

If you just want the quick-and-not-quite-nice version of the publically available, Continuous Deployment, DevOps, Microservices, Cloud, Disruption CV, then just go for something like TinyUrl, yielding something like:

https://tinyurl.com/jfornoff-cv

2. Using your own domain

Serving files directly from the Gitlab artifact under a nice URL usually requires you to run a HTTP server somewhere, redirecting or URL rewriting to your artifact URL.

You say that’s too much effort? I agree. After some digging around, I stumbled upon redirect.name.

It allows you to configure a DNS domain to use TXT entries for routing redirections, they host everything for you.

Assume you own greatdomain.com, and would like to have cv.greatdomain.com be the link to your hosted CV.

Step 1:

In this case, you would set a so-called CNAME record to make cv.greatdomain.com resolve to a redirect.name IP address (since we want them to take care of the redirection).

Two-sentence intro to DNS and CNAME records: - DNS associates (“resolves”) google.com (hostname) to 172.217.22.78 (IP address) - CNAME tells a DNS lookup “but our princess is in another castle - here is another hostname to resolve to get an actual IP address” (Sorry.)

Here’s how it would look:

cv.greatdomain.com.      <TTL>   IN      CNAME   alias.redirect.name.

Step 2:

redirect.name does not know how to redirect yet. We tell them in a TXT DNS record.

One-sentence intro to TXT records: There’s nothing to know, they just allow you to store some arbitrary text under a certain hostname. redirect.name uses it to read your routing definitions from there.

That would give you something along the lines of this:

_redirect.cv.greatdomain.com. <TTL> IN   TXT     "Redirects to https://gitlab.com/gitlabuser/my-cv/-/jobs/artifacts/master/raw/build/CV.pdf?job=compile-cv"

Now you can visit cv.greatdomain.com, and you get redirected to your downloadable PDF hosted in Gitlab!

Conclusion

I’ll be honest, this didn’t save any time, probably not even in the long run. However, it shows how to be deliberately intolerant to repetitive manual steps, which I personally consider a productive habit.

Hope you liked it and until next time!

Appendix

If I handwaved over something, you can find all the pipeline code for building a CV from LaTeX in my public CV repository.

Mix xref to the rescue!

Sat, 28 Oct 2017 10:15:04 +0000

Last week, i told you about the soon-included code formatter in the standard Elixir build tool Mix.

Today, we’re continuing our journey into the amazing things that Mix can do for you. There will be three features that I’m going to spotlight, just skip over whatever section if you already know it. This is a What the hell, this is a thing? How do I use that? type of post.

Short intro to mix xref

mix xref is a task that is able to analyze relationships between your Elixir modules. Sounds really boring, huh? It does. Let me show you why it’s actually amazing.

Finding dead end function calls

Did you ever fat finger a function name, run the code and get greeted with: HEY I’M YOUR RUNTIME ENVIRONMENT AND I TOTALLY CAN’T FIND THE THING THAT WE’RE TRYING TO CALL HERE?

This also assumes that you’re actually running the code locally (hope you have a good test suite that exercises your code base, otherwise… I feel sorry for you.) before shipping it. Anyways, besides the point. Imagine a world, where your compiler could tell you exactly that this is not going to work, quickly and for your entire code base!

Meet mix xref unreachable.

Assume the following project.

defmodule Greeter do
  def hello(name) do
    IO.put "Hello #{name}"
  end
end

Eagle eye, can you spot the error? Did you make one like these before? I have.

Now let’s see if we can get some feedback from our nice buildtool.

$ mix xref unreachable
lib/foo.ex:3: IO.put/1

That’s the short, machine-readable version (imagine if your editor would run this automatically and show you an indicator in the code whenever something like this happens! Hmm…)

$ mix xref warnings
warning: function IO.put/1 is undefined or private. Did you mean one of:

      * puts/1
      * puts/2

  lib/foo.ex:3

Ever wondered how these warnings are generated when you compile all your code? Yup. From this mix task.

Running only the tests you care about

Programming efficiently is a lot about feedback loops. The quicker you find out that the code you are currently writing does not quite do what you think it does, the quicker you can move on with your life.

Here comes mix test --stale. It works by keeping track of changes in your code since the last test run and determining the tests that could have been possibly impacted. Then it just runs those.

This become especially important when your test suite grows, maybe even spanning multiple sub-applications in an Umbrella app. It’s already well supported in TDD workflow tools such as mix test.watch, which supports a --stale flag.

Where’s that function used?

Ever stumbled upon a function that you really wondered what it does? Chances are that you’d be interested in where it is used, to figure out what parts of the code base are using it - to get some context.

It’s a lot like Jump to Definition, just the other way around!

$ mix xref callers Kernel.==/2
lib/example/compare.ex:84: Kernel.==/2

Neat! If you think this example is contrived, consider that you’re upgrading to a new major version of your favorite library and they changed some naming around. Run callers for the renamed function or module, walk through the call sites and make the necessary change. As soon callers says you’re clear, no need to wonder - can I commit this or do i need to double check again?

Conclusion

I’ll let you go now. Hope this post taught you something about how Mix can make your life easier or showed you a cool use of xref.

Until next time.

P.S. Be aware that mix xref is also able to output all the relationships between your application modules (e.g. in graphical format) - it’ll come back to you in case you ever need it.

Getting started with mix format

Sat, 21 Oct 2017 10:13:25 +0000

What’s code formatting and why do we care?

On my past job, I had the pleasure of writing quite a bit of Elm (which has the most outstanding developer experience that I’ve ever seen and is going to take over the frontend eventually, I hope).

One feature about the Elm workflow that really stood out to me, is that you: - Write some code that you think should work - Hit Save - Suddenly your reedit-problem-solving-jumbled code looks perfectly fine!

This does not sound like a big deal at first, but I invite you to try it for a day and see if you ever want to go back. And in my understanding, it’s not even about saved keystrokes, but about the mental context switches you’re not making.

OK, that works, time to make it looks nice, and then I can move on with the next step

Just imagine deleting this step from the equation. If you are a developer, I don’t think I have to argue to you that suspending your current train of thought does not help with solving the problem at hand. Therefore, I believe it won’t be hard for you to agree that reindenting lines is not what makes you happy or valuable to the world.

Code formatting in Elixir

Fortunately, José Valim and the amazing Elixir core team seem to agree with me and made code formatting a first-class citizen in the upcoming release 1.6 of Elixir.

At the time of writing this post, version 1.6 is not out yet. If you want to try it today, I recommend just running the master branch of Elixir.

Setup

For all Linux and MacOS users, I recommend using the ASDF version manager and it’s excellent Elixir plugin.

From there, you can run:

# Install the current master version of Elixir, to update just reinstall it
asdf install elixir master-otp-20

# Use edge Elixir as the global default
asdf global elixir master-otp-20

# Or just for the project that you're currently in
asdf local elixir master-otp-20

And have access to mix format, which runs the code formatter.

Format-on-save

In my book, this is the most important use case and the setup on your editor of choice is going to vary, but any self-respecting editor has some way of running commands on save or on a keyboard shortcut.

There’s even some plugins that take that configuration from you: - Atom - Neoformat for VIM

Side note If you happen to set this up by yourself, most formatting frameworks support piping the file contents to the formatter via STDIN and reading the formatted output from STDOUT, which makes for snappier formatting. This is supported by mix format -.

Whole-project formatting

To do this, you can either pass a pattern like:

mix format "lib/**/*.{ex,exs}"

However, I’d recommend adding a .formatter.exs file to your project root (that’s where mix format is looking for it by default, but you can put it wherever you want. That’s what she said.):

[
  inputs: [
    "mix.exs",
    "{config,lib,test}/**/*.{ex,exs}"
  ]
]

This enables you to just run mix format with no arguments to reformat all the code in the specified paths.

That’s all I got for now, hopefully my little essay persuaded you to give automated code formatting a try, if not in Elixir, then maybe in your language of choice. Until next time.

Elixir Macros Part 2 - How to find out what's going on

Sun, 15 Oct 2017 10:07:32 +0000

In part 1, we looked into how Elixir macros work and how they allow us to write code that writes code, as many say.

One drawback of using macros is that they are hiding complexity, making it less obvious what exactly a given piece of code is doing. Today, we’ll figure out how to dive into libraries and track down what macros are doing under the hood.

To make this as realistic as possible, we’ll use an example from real library (Plug). In a nutshell, Plug provides abstractions to deal with HTTP requests and responding to them.

Whirlwind introduction to Plug

To understand what Plug does, you just need to understand two concepts. Firstly, the concept of a Plug.Conn to represent an incoming request being processed, and secondly the Plug itself, which represents a processing step on a Plug.Conn.

The Conn data structure

When you consider a HTTP request entering your application, there is a bunch of data that you care about. Starting from the HTTP verb (e.g., GET / POST), over HTTP headers (e.g. Authorization to check which user made the request), to the request body.

%Plug.Conn{} is an Elixir struct that encapsulates all this data. Processing of requests is then done by routing this data structure through a sequence of processing steps (those steps are called Plugs), fabricating the response.

Plugs as a way of transforming the Conn

As we just touched on, the %Plug.Conn{} also contains data about how the HTTP response should look like, e.g., the HTTP response code and response body.

A Plug is literally just a definition of a transformation step, either in module form (ModulePlug.call(conn, opts)) or function form (functionPlug(conn, opts))).

Now let’s imagine a MVC scenario where an incoming request hits a router, gets dispatched to a controller action and eventually is answered by the server, maybe with a rendered HTML view or JSON response.

Conceptually, the running web server implementation will receive the HTTP request, wrap it into its %Plug.Conn{} representation, and pass it to the router implementation given in it’s configuration.

Now for the really interesting part: This router is a Plug. It receives a %Plug.Conn{}, transforms it and returns a %Plug.Conn{}. That’s all that Plugs ever do. Calling our router might look as simple as:

AppRouter.call(conn, additional_router_options)

The router may then, as a part of its processing, match on the request’s HTTP verb and path and delegate to a controller. Guess what that controller is. Yuuuuuup, surprise, it’s a Plug.

Congratulations, you now know everything there is to know about Plug.

Deconstructing the Plug router

Here’s a very simple router definition (borrowed straight from the excellent Plug.Router documentation).

defmodule AppRouter do
  use Plug.Router

  plug :match
  plug :dispatch

  get "/hello" do
    send_resp(conn, 200, "world")
  end

  match _ do
    send_resp(conn, 404, "oops")
  end
end

I won’t insult your intelligence by reiterating what this router does. Rather, I’ll tell you how to figure out how this code functions.

What’s that `plug` thingie and how does it exactly work?

At the time of typing these words, I don’t know either. Let’s find out.

To start, we might wonder if the plug keyword is part of Elixir itself (Spoiler: It’s not.). Excellent. Apparently this is a macro. The next question we might ask is: “Where is this coming from?”

Generally, macros cannot just appear out of thin air, Elixir doesn’t allow for this, because it’s intransparent.

The only two ways that macros may enter a module is as follows:

require:
This allows us to use macros defined in the required module in the form of:

require Logger

Logger.info "Hello world!" # `info` is a macro defined in the `Logger` module

import:
This merges the imported module’s functions and macros into the scope of the current module. We can then call macros like:
```
import Logger

info "Hello world!" # Note how we don't need the `Logger` prefix!
```
use: This is our special candidate. It does not directly make macros available to our module, but it can include require or import statements!

Super tl;dr version of what this statement does: It invokes the __using__ macro in the used module, so its just the syntactic sugar equivalent to TheModule.__using__. Mostly used to dump some functionality into your module (Notice how we’re using Plug.Router above!)

Having established what to look for when hunting macros, let’s go back to our router example. We were trying to figure out where the plug macro is coming from, remember?

The only line that may inject macros is obviously:

use Plug.Router

Diving into the Plug.Router source code, we find:

# Some details omitted for brevity
defmacro __using__(opts) do
  quote location: :keep do
    import Plug.Router

    use Plug.Builder, unquote(opts)

    def match(conn, _opts) do
      do_match(conn, conn.method, Enum.map(conn.path_info, &URI.decode/1), conn.host)
    end

    def dispatch(%Plug.Conn{assigns: assigns} = conn, _opts) do
      Map.get(conn.private, :plug_route).(conn)
    end

    defoverridable [match: 2, dispatch: 2]
  end
end

Hmm, no defmacro plug… OK. Repeat the process. We got two new candidates! 1. import Plug.Router 2. use Plug.Builder

Glancing over the functions in Plug.Router, no hits. Moving on.

Checking on Plug.Builder.__using__:

defmacro __using__(opts) do
  quote do
    # Again, shortened for clarity

    def init(opts) do
      opts
    end

    def call(conn, opts) do
      plug_builder_call(conn, opts)
    end

    import Plug.Conn
    import Plug.Builder, only: [plug: 1, plug: 2] # <- OHHHHHHHH BABY

    Module.register_attribute(__MODULE__, :plugs, accumulate: true)
    @before_compile Plug.Builder
  end
end

Jackpot! So plug is defined in Plug.Builder!

Go deeper you say? Hell yeah! On to the plug macro definition.

defmacro plug(plug, opts \\ []) do
  quote do
    @plugs {unquote(plug), unquote(opts), true}
  end
end

Apparently, calling plug just sets the @plugs module attribute to a 3-item tuple. So what’s that about?

We could technically wrap up here, we found the macro definition!

via GIPHY

But where’s the fun in that?

How Plugs can be composed out of other Plugs

I’d like you to go back to the Plug.Builder.__using__ macro for a second.

Notice the two last lines:

Module.register_attribute(__MODULE__, :plugs, accumulate: true)
@before_compile Plug.Builder

To shorten this a little bit, the call to Module.register_attribute states that we want our module to not reassign @plugs when we repeatedly assign it but instead add the new values to an accumulating list.

The @before_compile module attribute is reserved by the language and hints the Elixir compiler to invoke the __before_compile__/1 macro in Plug.Builder before compiling the module and put its output at the end of the router module definition (for curious people).

Here we go, what’s in the __before_compile__ box?

defmacro __before_compile__(env) do
  # Apparently we're grabbing all the defined plugs from the @plugs list
  plugs        = Module.get_attribute(env.module, :plugs)
  builder_opts = Module.get_attribute(env.module, :plug_builder_opts)

  # Oh look, it's compiling all our plugs into something new!
  {conn, body} = Plug.Builder.compile(env, plugs, builder_opts)

  quote do
    # Defines a private function on our router module, not too bad
    defp plug_builder_call(unquote(conn), _), do: unquote(body)
  end
end

OK, everything is straightforward. Just Plug.Builder.compile/3 seems crazy, fasten your seatbelt, we’re going in!

@spec compile(Macro.Env.t, [{plug, Plug.opts, Macro.t}], Keyword.t) :: {Macro.t, Macro.t}
def compile(env, pipeline, builder_opts) do
  conn = quote do: conn
  {conn, Enum.reduce(pipeline, conn, &quote_plug(init_plug(&1), &2, env, builder_opts))}
end

To keep this post somewhat manageable, we are going to handwave just the tiniest bit and just give you the big picture.

Our router module calls the Plug.Builder.__using__ macro, which defines call/2 on it (routers are Plugs, remember?), which just delegates to plug_builder_call.

To define plug_builder_call, we are running over all the Plugs filled into the @plugs module attribute (e.g, using the plug macro that we examined) - those are in the pipeline parameter of Plug.Builder.compile.

From there, we are just nesting the Plugs in one another, effectively ending up with something along the lines of this:

defmodule AppRouter do
  def call(conn, opts) do
    plug_builder_call(conn, opts)
  end

  defp plug_builder_call(conn, opts)
    conn
    |> match(opts)
    |> dispatch(opts)
  end
end

The call/2 function now just composes the other Plugs defined within it, in the order of appearance. Neat, huh? It’s like a chain of transfomation steps that our request is handed through!

The match and dispatch function plugs are defined in Plug.Router.__using__, but this post is already 1400 words long, so we’ll wrap it up here :-)

Until next time!

Elixir Macros Part 1 - How they work

Tue, 03 Oct 2017 11:59:53 +0000

TL;DR Elixir is nice. Macros enable you to create a concise language for the application problem domain without applying magic.

Today’s post will be about Elixir macros, mostly just documenting my own learning process by retelling the story.

Why do we care?

Programming languages are used to solve all types of problems. Web servers, games, banking applications, you name it. We appreciate low-boilerplate code that caters to the problem domain. Conciseness and readability are what we’re looking for. However, it is impossible to support expressiveness for all those use cases in the core language. Therefore, we need a mechanism to create readable code. Creating such domain-bound abstractions involves hiding frequently unimportant details. We need to put these details away while making sure that it’s very clear where to find them. This makes code maintainable and simple to reason about.

Code as (transformable) data

Let’s take a step back. The first important idea to understand what Macros are used for is where they sit in the compilation process. When you compile Elixir code, the text of the source file is parsed into an Abstract Syntax Tree (AST), which represents the code in syntactic form.

Simple example:

1 + 1

is represented in the AST as:

{:+, [context: Elixir, import: Kernel], [1, 1]}

This is a tuple with 3 elements, the first being the name of the function being called, the second metadata, and the last the list of function arguments.

What role do Macros play?

After generating the Elixir AST from the source code files, Macros are expanded by the compiler. That means: When the compiler finds a call to a Macro, the compiler executes it and substitutes the result of the Macro invocation into the AST.

Therefore, Macros need to return an AST representation, such as the one we saw in the example. Our core takeaway from this is that Macros just return data (in this case a data-representation of source code).

Let’s take a look at a fun little Macro.

 1defmodule MyLogger do
 2  @enabled true
 3
 4  defmacro log(message) do
 5    if @enabled do
 6      quote do
 7        IO.puts unquote(message)
 8      end
 9    end
10  end
11end

This is a logger module whose log/1 Macro you can use, and can turn off all log output using @enabled.

How would that work in practice?

require MyLogger

# when @enabled is true
MyLogger.log("Hello!") # => :ok
"Hello!"

# when @enabled is false
MyLogger.log("Hello!") # => nil

Note the calls to quote and unquote, those are the bread and butter of writing Macros. For the following explanations, I’d invite you to think of writing Macros as being analogous to the concept of string interpolation. The block passed quote outlining the string (or AST-fragment if you will), unquote used to insert predefined values.

Line 5 (if @enabled) is executed at compile time, whenever the Macro is expanded. Therefore, if @enabled is false, the quote-block will not be returned, thus defaulting to nil and inserting nothing into the Macro-site AST.

If @enabled is true, quote states that all code inside of it will be returned as the AST data representation. The effect of this is literally as if you copy-pasted the code inside the quote block into the module where you called the Macro.

Now note line 7, specifically unquote(message). If we did omitted unquote(), the message value from the function argument would’ve not been inserted into the AST, but instead it stays as a “placeholder” that is evaluated as soon as runtime execution passes that line. For example, it might output the result of the function message() of the module that we compiled the Macro into.

Conclusion

We can conclude that expanded Macros yield some code representation that is then injected into the AST representing the Macro-calling module. When we want to inject values into that code representation (e.g, Macro function arguments), we have to use unquote. Otherwise the code is just executed in exactly the way we write it in quote.

References

If you’d like a much longer and more in-depth resource on Elixir Macro mechanics, I highly recommend Saša Jurić’s series of blog posts on Macros!

Up next

Macros are constructs that make code and the origin of functionality slightly less obvious because they inject code into your modules AST. This is intended and necessary to create more concise and succinct code, but makes it more challenging to track down how things actually work.

Therefore Part 2 will be about Macro Archaeology and how to dig up functionality if you want to dive in the nitty-gritty hidden details.

Cold showers every day

Thu, 24 Aug 2017 08:00:00 +0000

A while ago, I read this article by Kyle Eschenroeder. It talks about some heuristics in striving for a fulfilled life and some pitfalls you might want to avoid in today’s society, where you can essentially pursue whatever you want.

Kyle introduces the concept of eudaimonia, which proposes acts that might cost some extra effort in the moment, but leave you feeling good about yourself later on. These acts are also very much about bettering yourself as a human. Getting up earlier to work out might not feel great while you’re struggling to crawl out of bed, but as you leave the gym, it will leave you with a sense of satisfaction — I did well today.

This post is about such an eudaimonic deed. Taking cold showers. First thing in the morning, every day. How the hell is freezing my ass off going to help me be a better human being?, you might ask. Great question. Here’s why I think it is a good idea.

Wakes you up. Thoroughly.

No 2-hour wake up stagger until you hit your stride. This synergizes nicely with trying to build a habit of getting up earlier! You might as well stay in bed if your brain-fog prevents you from doing anything useful.

Requires you to step outside your comfort zone.

This in itself is worthwhile, but you also want to practice getting out of your comfort zone over and over. Getting it done first thing in the morning allows you to:

Form a habit very easily!
Not be as tempted to skip it, because it sucks (it does).
Most people tend to not think a lot right after getting up, this helps.
Health benefits: There probably are some, but I don’t even care.

That’s it for my first post.

About

Mon, 01 Jan 0001 00:00:00 +0000

Hey! 👋🏻

My name is Jan. Once upon a time, I finished a Masters Degree in Business Information Systems at TU Darmstadt. I’ve been writing software for quite some time, did 6 years of Ruby working while studying and 2 years of Elixir. Functional programming is something I grew to enjoy a lot, since it fits my approach to problem solving and seems to keep complexity at bay, mostly. Here are some of my side projects.

The operational side of software and making things work really well when the rubber hits the road is very enjoyable to me. I’m also sort of a performance nerd in that way, I love benchmarking and optimizing.

During my university career I touched a lot of networking technology and I’m really interested in Distributed Systems, how they operate and thinking through the cases where things can go wrong.

Previously, I’ve been working and learning alongside the great people of Grandcentrix part-time while freelancing on the side. In September 2019, I joined Google Zurich as a Site Reliability Engineer.

Drop me a mail if you think we should have a chat!

Also my CV is here (or you can find it in the navigation bar).

Imprint

Mon, 01 Jan 0001 00:00:00 +0000

According to §5 TMG:

Jan Fornoff
Amselweg 66
64295 Darmstadt

Email: janfornoff@gmail.com

Projects

Mon, 01 Jan 0001 00:00:00 +0000

Gitctx

CLI tool (written in Go) that allows to have multiple profiles of Git committer information (i.e., name & email address). You can then configure on a per-project basis which of these profiles to use.

Standupbot

Bot that allows you to have asynchronous standup meeting via Slack DM. Standups are formatted and posted to a status channel for team members and managers to see. Every user can configure his standup time independently.

Meetup presentation
Written in Elixir using OTP
Deployed to multiple Slack workspaces and in active use every day

Multiplayer TicTacToe

Written in Elm and using Phoenix Channels ❤️

Written for a workshop at the Elixir RheinMain Meetup
Got me started with Phoenix Channels and handling Websockets in Elm
Source code available: Elixir backend & Elm frontend

Game of Life

Written in Elm ❤️

Implements Conway’s Game of Life
Taught me quite a bit about encapsulation in Elm modules and how to represent the current Game state as Elm types
Source code available on Github

Chefkoch-Shopping

Experiment while checking out Node.js

Integrates with the Wunderlist API
Fetches Recipes from Chefkoch
Parses ingredients (Chefkoch does not offer an open API, unfortunately)
Allows selecting and editing ingredients to insert into Wunderlist

Jan Fornoff

Consistent Hashing

The Problem

Request hashing

Ohhh no, elasticity!

Consistent hashing

The Visualization

Conclusion

On Circuit Breakers

Hit me baby, one more time

Why the name?

Meditating on failure remediation

Conclusion

Introduction to the Transmission Control Protocol

Bird’s eye view

Buffering

Conclusion

Introduction to the Internet Protocol

What this post is not about

What this post IS about

Service: Giving and taking

Guarantees

Conclusion

Book Notes: "A Guide to the Good Life"

Tranquility & Virtue

Adversities

Negative visualization

Choosing your battles

Introduction to Linux sockets

What are Sockets and how are they created?

Communication domain

Socket type

Stream sockets (SOCK_STREAM)

Datagram sockets (SOCK_DGRAM)

Socket protocol

Connecting Sockets

Server side

Client side

Summing up

Documentation Driven Development - or why names matter

Problem solving is verbal

Document to establish vocabulary

Conclusion

Continuous deployment for your CV

Frequently Asked Questions

The Plan

Automatic building of PDF

Making the PDF publically available

Getting a pretty URL

1. URL shortener

2. Using your own domain

Conclusion

Appendix

Mix xref to the rescue!

Short intro to mix xref

Finding dead end function calls

Running only the tests you care about

Where’s that function used?

Conclusion

Getting started with mix format

What’s code formatting and why do we care?

Code formatting in Elixir

Setup

Format-on-save

Whole-project formatting

Elixir Macros Part 2 - How to find out what's going on

Whirlwind introduction to Plug

The Conn data structure

Plugs as a way of transforming the Conn

Deconstructing the Plug router

What’s that plug thingie and how does it exactly work?

How Plugs can be composed out of other Plugs

Elixir Macros Part 1 - How they work

Why do we care?

Code as (transformable) data

What role do Macros play?

Conclusion

References

Up next

Cold showers every day

Stream sockets (`SOCK_STREAM`)

Datagram sockets (`SOCK_DGRAM`)

What’s that `plug` thingie and how does it exactly work?