The state of modern Rx in iOS

Thanks to undraw.co for a great illustration

This article focuses on different limitations of the Reactive Extensions implementation of FRP paradigm.

Reactive Extensions is a family of frameworks that allow for functionally-reactive programming in imperative programming languages. One of the greatest factors that allow us to treat all of the implementations as a part of one family is the fact that all of them bring identical concepts to the table, which is reflected in similar primitives naming, operators semantics and life-cycle control tools. The advantages of x for commercial projects include:

1. Seamless asynchronicity
2. Transformation atomicity
3. Use-case configurability

Right now we will, however, take a tour on 3 aspects of Reactive Extensions that showcase the limitations of the modern popular implementations.

1. Error event typing and transformation
2. Scheduling and synchronicity context preservation
3. Static allocation

In this article, I will be providing the samples related to the iOS platform and the ReactiveX/RxSwift implementation, in particular, because my primary commercial experience is with this exact environment.

Error event type information and its transformation

The debate of whether if an Error event should be generically typed and its type preserved by the reactive primitives exists since the dawn of .NET implementation. It is a completely defensible position that the presence of typed errors, just like many things in commercial and academic programming, is utmost a matter of preference and there is no such thing as quote-unquote the right answer.

Notably, this discussion stems from the same preferential semantics as a discussion for typed exceptions for your favorite language. For instance, Java features full-blown support for explicit exception types specification. And then, a throws annotation that allows you to specify these very conditions that are used to type-check your code and generate documentation might be right up your alley.

Conversely, one may find it interesting that quite a few modern languages featuring very detailed and thorough static type systems, such as Swift and TypeScript don’t carry the ability to specify the type of a thrown exception out of the box. Ironically, even Kotlin, a language that has its first runtime ever developed be JVM-based, does not feature support for typed exceptions.

I would like to illustrate the problem with a code sample. NB: the following is trivial just like the problem itself and if you have written at least a bit of commercial Rx code, you will not find anything new here for yourself, might as well skip ahead.

Given these signatures and knowing that both of the functions’ resulting observables may emit errors you have two choices.

One choice is to allow errors to fall through the execution context in their initial forms. This is easier to do, and is somewhat induced by this very same practice being applicable to regular exceptioning. Similarity is there especially with the older languages that don’t require explicit throwing semantics acknowledgment, such as C#, Java, JavaScript, where you might as well treat every other function as a function that throws.

Previously, you called a function that never thrown anything, but momentarily its implementation got changed, but neither your call nor any other call above draws any information about the new behavior of the innermost function. If any situation like that has a potential of occurring, then you probably should get ready for exception handling earlier, while writing the whole chain of calls. More modern and powerful languages such as Kotlin and Swift force a concept of explicit exceptioning semantics requiring you to mark and handle/explicitly propagate all the calls that can throw.

So yeah, the code where the errors fall through will look like this:

The business rules in commercial programming, however, often suck and make programmers do a lot of unwarranted useless work. One of these might be for instance a requirement to treat different errors in different ways. What if there is a common widget that illustrates all the wrong comings of the network activity, but we don’t expect the parsing errors to be there? And what if there is more than one error that the download process may emit — one of the errors is TCP negotiation failure, and another — 401 HTTP status code, indicating invalid auth? This all becomes rather finicky to work around with our latest approach, which brings us to the concept of typed errors. I am going to write my own microscopic implementation of Rx’s Single to illustrate this concept.

These are going to be our signatures that we use. Note that the download(from:) function returns a Single with a very complex error type. The usage of these will be looking like this:

After you subscribe to a Single returned by this function, determining what exactly has gone wrong in the process becomes a breeze. The source code is still here.

Fun fact: Apple Combine has typed errors, which is another can of gas straight into the Rx fire.

Scheduling and synchronicity context preservation

Take a look at this sequence of declarations.

You know, the types of foo, bar and baz are identical here. From the static typing perspective the only thing we know about these observable sequences they are regular observables that emit Void-type next events and the Swift’s standard library Error-type error events. You may or may not guess that those are not quite all the characteristics of these observables. The two more we are missing are:

1. Whether if an observable sequence will emit 1 or more events synchronously upon subscription.
2. Which thread is it that emissions will appear on.

Let’s go and characterize all these observables by these criteria which are not reflected statically:

1. Plain just creational operator. Will definitely emit all of its values synchronously. It is not bound to particular scheduling.
2. Just with an observeOn with a concurrent dispatch queue scheduler. Forwards every single event to a deliberate concurrent dispatch queue. Will definitely make all of the emits asynchronous.
3. Just with an observeOn with the potentially synchronous MainScheduler. Will always emit on the main thread. Will emit synchronously if the execution context is already on the main dispatch queue, will schedule for asynchronous execution only if it isn’t.

Actually, for that matter, let’s get back to the first observable because come to think of it there are ways to screw it up even worse. You can use the create operator to have full control over the observable’s execution context and you are theoretically capable of emitting events on random threads, lol. These shenanigans will be handled on any rescheduling, but without the synchronization code there is not way for RxSwift to ensure the validity of this concept of atomicity of each emit handling. And if you are to implement your own version of an observable at the lowest level by default you allow your user to create custom observables that go all over the place in terms of calling the subscription handler for several events at the same time on separate threads of execution.

So yeah, so similar yet so different. The lack of opaqueness leads to all sorts of funny stuff. For instance, you can use the combineLatest creational operator with two observables, one of which will emit on main, the other one — not on main.

Do you know which thread will the items be received on? Well, me neither, so I have written the code. Apparently, if any of your observables last known scheduling was MainScheduler, RxSwift assumes that you would like to receive all of the events on the MainScheduler. And then I remove the MainScheduler and replace it with another, unrelated serial dispatch queue scheduler. RxSwift goes the fastest way about working with those and doesn’t pick any particular scheduler for the consequent events. My latest update has arrived on Thread #3 that I have myself created. Then, the other observable emits a new event on its own scheduler, rounds up as Thread #6 on runtime, and suddenly the handler in subscribe is invoked from the Thread #6.

Alright, so let’s implement another concept of how can we make observables be more informative in these concerns. Won’t bother with completion and error event because it’s out of focus here. We will also make the Disposables RAII exclusive in that there is no dispose method and you perform the resource clean-up and cancellation just by releasing the Disposable, with your custom logic being run on deinit. Here’s our plan:

1. See the new ScheduledObservable in comparison to a regular Observable
2. See all the available schedulers and their families
3. Look at the combineLatest operator implementation and see how it leverages the all new static type infromation.

ScheduledObservableType

We start with an Rx-esque declaration of a regular observable:

And then we introduce a new type, the ScheduledObservableType, which adds the new type information on top of the regular observable.

Notably, we do not utilize the protocol inheritance for this declaration and instead stick to composition. This way we are avoiding the operator declarations clash so that you don’t see a map operator twice, one being for the regular observable and the other one — the one that retains the scheduling information. Ugh, what else, what else… Oh, right, given a scheduled observable we can always demote it to a regular observable by touching the unscheduled getter. You aren’t expected to use the framework that way, however it has its uses in the internal transformations and stuff.

Alright, let’s dig them schedulers.

Regular observable doesn’t have one. What this means is that

1. The observable is free to emit 0+ items synchronously upon subscription
2. The observable is free to emit 0+ items asynchronously during the subscription’s lifetime
3. Any asynchronous event is free to be emitted on any DispatchQueue desired, without any similarities between events.

That is the worst case scenario and none of the schedulers of the scheduled observable support this variety of outcomes.

Schedulers’ traits

The schedulers have traits which are expressed using the protocol inheritance. This allows us to gather the schedulers into families and treat them commonly.

Here’s one:

Nuff said: the schedulers that implement this trait exist only in a single instance which is accessed via an instance static getter.

Here’s another one:

Fairly straightforward too: the schedulers that implement this trait feature an dispatch queue under the hood.

And here’s another one:

These schedulers support being synchronized against each other i.e. having their emissions happen mutually exclusively.

And here’s another, the last one:

The schedulers that implement this trait can be reproduced featuring equivalent semantics. The only scheduler that implements this trait is the one that works with serial dispatch queues. The framework expects that if you used a serial dispatch queue scheduler, you don’t actually care which queue will it be exactly on which the emission will occur so we sometimes can create one additional scheduler to synchronize two different ones.

Schedulers

Let’s take a look at the available schedulers and what they mean.

1. AllSyncScheduler: SingleInstanceScheduler, SynchronizedScheduler

This scheduler represents that every single event emitted by an observable will be emitted synchronously upon subscription. There is only one instance of such scheduler per process. The creational operators that produce observables with such scheduler include just, from and of.

2. MainScheduler: KnownSchedulerType, SingleInstanceScheduler, SynchronizedScheduler

You get an observable with this scheduler by calling observeOn operator.

In RxSwift semantics, that only one instance of this scheduler should be statically referrable per asyncInstance instead of instance. The framework doesn’t feature an implementation of observeOn that detects that the current DispatchQueue is already main and if so, schedules the emissions synchronously.

3. SerialDispatchQScheduler: KnownSchedulerType, ReproduceableScheduler, SynchronizedScheduler

We can reproduce this one for synchronization purposes because as I said earlier, the framework user should only care that it’s serial, but not which exact queue is it that the emissions arrive on.

4. DispatchQueueScheduler: KnownSchedulerType

Can’t really do any meaningful synchronization with it. You can feed any queue you like there, but the framework assumes that the queue you passed there was concurrent.

External synchronization primitives

I swear to god, I am rewriting this part of the article for the third time. We will occasionally need a full blown mutex for external synchronization in cases where observables are not synchronized with each other or they are inherently impossible to synchronize. Here is my concurrent-dispatch-queue-based totally-not-copy-pasted-from-bark-overflow implementation of a mutex.

We will also have a MutexUnsafe class that has an identical interface minus the dispatch queue and its usage. Useful for polymorphysm.

Combine Latest operator

The combine latest operator is a stateless operator with a state machine that roughly looks like this.

The algorithm of combine latest subscription over N observable works as follows:

1. Create the state featuring N slots for each subscription’s emissions
2. Each slot is unitialized
3. Subscribe to all N observables synchronously
4. On each emit, save the event into the corresponding slot
5. If the state features all initialized latest events for all N subscriptions, forward them to the transformation predicate
6. Emit the result of transformation up the subscription chain

Needless to say, if two subscriptions emit on 2 separate non-main dispatch queues, this may cause concurrent read/write access anomalies. In such cases we will need to provide synchronization externally, using that Mutex I’ve showcased earlier.

The disposable, among the source subscriptions, will also carry our state machine. It’s also generically typed to use some implementation of MutexType protocol wrapping the state machine.

Here’s the code of the resulting observable’s subscribe method:

We’re all set. Let’s utilize our new type information to create accordingly optimized versions of the combine latest operator for public usage.

First, let’s cover the regular unscheduled observable. No type information means that there can be the worst case scenario of both observables emitting in different threads and we will need to use a full blown mutex to make them accesses safe. Additionally, the type doesn’t get converted to a proper scheduled one because why would it lol.

Then let’s go for the best case possible:

1. Both observables’ schedulers are equal to each other
2. Both schedulers implement ReproduceableScheduler trait
3. Both schedulers implement SynchronizedScheduler trait

There is only one scheduler that is reproduceable and synchronized — it’s the serial queue scheduler. In the case of one such there are 2 possible roads we can go:

1. The two observables’ schedulers’ dispatch queues are referentially equal hence are the same serial queue hence all emissions will be happening only one at a time hence no external synchronization required.
2. Dispatch queues are not equal. In this case we will reschedule each observable to a reproduced scheduler, which will be essentially another serial dispatch queue, which will cause the synchronization to appear hence we won’t need external synchronization either.

Here’s the code:

No need to be intimidated. Just keep in mind that combineLatestUnsafe accepts and returns unscheduled observables. So we unschedule the initial observables. After it, we perform an internal operator called promoteToScheduled which just promotes the observable to a scheduled type without touching the actual scheduling at all. Internal, not for production use.

After this we jump to the case where:

1. Observables’ schedulers’ types are equivalent
2. Schedulers inherit SynchronizedScheduler trait
3. Schedulers inherit SingleInstanceScheduler trait

It’s pretty much more of the previous implementation except since they are single instance schedulers no need to dynamically compare shit. Them is synchronized? Use an unsafe mutex AKA no external synchronization.

Okay, that was the third, and now we go for the last common case where:

1. Schedulers might as well be different
2. Schedulers might as well be not synchronized
3. Schedulers are not reproducible
4. Schedulers be of these types that allow multiple instances

For the lack of static type information we solve this case by externally synchronizing all of it.

But hey — here’s the catch. The observable type gets denoted back to an unscheduled observable because we’ve got not a single idea about how it’s going to be scheduled in the end.

Usage

I’ve — just dumped it all into a single gist.

So that’s it. We’ve taken a look at how we use this all new type information for the internal design and optimization. However, the real jam in this whole concept comes one we start to utilize it in the architecture of our APIs.

API design utility

UIKit’s a good kid, am I right? Every single view’s state mutation needs to be performed from the main thread. And we can reflect this nature actually in our design. Here’s one: there is not only scheduled observables but also scheduled observers.

Given an instance of UILabel we can use instance.rx.text to reactive bind some stream of strings to the label’s text value. Consider this bit:

And oh boy — we are leveraging the type system, making sure that you can use this observer to subscribe exclusively only to observables that emit on the main scheduler!

Surely you already see where this is going.

Yep, this whole combine latest operator implementation is written just for sake of explaining this type system and showing these 2 jpegs of code.

Please go take a look at the PastFive framework implementation to get the whole picture.

Subscriptions’ static allocation

The whole thing I am going to be talking about in this section is totally disconnected from commercial development. Not ever would be useful in real like and it’s there essentially to just fiddle around. In a proper Darwin application this debate is irrelevant for many reasons:

1. UIApplication, which needs to be instantiated on process start, is an NSObject. Objective-C doesn’t support statically allocated NSObjects.
2. A lot of APIs in AppKit and UIKit are not as static as they appear. One such example I’ve seen while trying to UIScreen.main.bounds before applicationDidFinishLaunching.

So ugh, that Disposable protocol, right. We didn’t constraint it to be specifically implemented only by classes. But you will be using classes if you want to rely on RAII semantics to manage the subscription’s lifecycle. And also remember that “combine latest” state machine we’ve implemented in the previous section — like, with it you won’t appreciate if this state was suddenly copied once you call disposed(by:), right? Here, dynamic allocation is inevitable despite our best attempts to avoid it. These attempts include the fact that Observable and ScheduledObservable are both in fact structs. We would also use the dynamic allocation for our PublishSubject or BehaviorSubject I could be bothered to implement one lol. Oh come on, even share operator would capture a mutable array of subscription handlers in the dynamic storage of the closure.

But what I can easily imagine is how much use there would be of a proper reactive framework while working in an environment where there is only stack-based allocation available. We will implement a generically typed just creational operator and its disposable. Source code can be found here.

To begin with, we will require a stack-allocated closure for two cases:

1. Implementation of the subscribe method of an observable
2. Implementation of the observer that will be used in the subscription to the just operator.

A closure in C

First, the type that represents our closure:

This macro-based closure is a struct that will have whatever typedef NAME its given. It contains 2 things. The first one is the pointer to function that accepts 2 parameters: the parameters of the whole closure and the closure’s context (or, in other words, it’s capture list). We’ll have a helper function also defined inside the macro that abstracts the calling of this closure.

We prepend call_ to whatever NAME the macro user gave us, making it no less convenient to use, but lol at least the whole logic is now located in one single place.

Also, let’s declare a function that will act as convenient initializer for a given closure.

Straightforward as can be. As you can see, we append _init to the NAME macro parameter to declare this function. What is this, C macro programming tutorial? Alright, we’re done here with this thing, here is a quick usage example.

I certainly hope you dig.

Observable created with just operator

To kick things off with this bad boy we should remember that this observable is supposed to emit only once and synchronously. We can cut a corner with our disposable implementation and make each and every single instance of subscribe call produce the same pointer to a function that accepts void, returns void and does nothing because that’s what it’s supposed to do. Note that we don’t even need to simulate generic programming by using macros for this one.

And there goes the easy part. First, we need to describe what does out event handler for this observable look like in a generic sense. We will, of course, make it a closure.

Given an event type name string this will generate us a handler closure signature called just_handler_closure_string_t.

Then, we need to generate the signature for the subscribe call.

Here, the closure_ctx struct features a single member called captured which will be initialized with the value passed to the just creational operator. If we had object-oriented programming support and a semantic this, we could assign this captured event to this and access it in the subscribe to capture it in the factory and use for emission by passing it to the handler. If we had proper syntactic closures we would’ve written even less code, simply capturing the parameter in the factory immediately in the just creational operator call. Instead, here we will do eventually the same thing except allocating and initializing the captured context explicitly.

Here is the core of the whole observable — inner content of the subscribe closure. Don’t really see much difference between this and Swift except for shitty naming.

As you can see, after the emission it returns that pointer to the disposable implementation that is always the same.

Now, for the actual observable.

The just_observable will have a single member, which is a closure, which you will call get yourself a disposable. You call the disposable — you cancel subscription. The _init allocates the observable and the ctx for the subscribe call, filling the captured value with whatever was passed into this whole creational operator-function-thing. And then we initialize the subscribe closure. The lambda implementation of the subscribe closure, the just_subscribe_closure_lambda_##EVENT_TYPE_NAME, as seen before, upon being called reads the ctx and emits the value in there only once to the handler passed upon subscription. Very basic stuff.

The usage of this mess

First, we declare the observable and everything related for the given type.

CENERIC_JUST_OBSERVABLE(int)

This will give us a set of functions related to this type that work with all the inner parts of the observable creation process. So, we will need to define a function that will handle the emissions of our subscription.

void handleThemInts(just_handler_closure_ctx_int_t* ctx, int event) 
{
}

Then we can create our observable and the handler. Handler is a closure and will require some empty ctx to be created. The observable is created with a simple function call.

just_observable_int_t observable = just_observable_int_init(4);
just_handler_closure_ctx_int_t ctx = {};
just_handler_closure_int_t handler =
    just_handler_closure_int_t_init(ctx, handleThemInts);

And then we can subscribe, and cancel afterwards.

just_disposable_t cancel =
    call_just_subscribe_closure_int_t(
        observable.subscribe, handler);
cancel();

Phew. Well, that’s a lot of hassle. You can find all the source code over here. You can even notice how I didn’t even bother to use the Carbon to generate the pretty code for this last part because there really is nothing too pretty about this darn thing. You know what, I planned to do more for this whole third part, but now after writing this I am just realizing what a great hassle it is gonna be. So many things wrong with it:

1. We will have to actually manipulate separate disposable types for each and every single observable
2. Calling the closures and building contexts is so cumbersome
3. I didn’t even touch on the topic of actual asynchronous programming, which is btw one of the most important parts, duh
4. The syntactical organization of the operator chains is quite a question to be asking
5. The codebase that utilizes macros so much is an ass to work with
6. The lack of interfaces and associated polymorphic features doesn’t help at all
7. Combine latest operator will generate so much code — ooof.
8. Any stateful operator for that matter — ooof.
9. Subscription sharing — well, as long as you reserve static slots for your subscriptions…

Actually, this whole third part makes me think, damn. A great chunk of the kernel programming is comprised of dealing with these 20+ years old codebases written in deliberately poor, syntactically weak languages. If we would measure the “goodness” of a programming language by the range of different smart syntactic tools available, the C goes straight into the no-fun zone, where it’s easy to just get to the situation where 95% of your development is more like dancing around the limitations of the language. As this one guy from JetBrains told me on this one Russian conference: a good starter programming language teaches you how to solve problems, the bad one teaches you how to dick around passing mutable objects by reference.

Gladly, this wasn’t a huge waste of your and my time. At least we now can have a glimpse of just how much expensive it is man-hour wise. This is not a commercial programming thing. So I am just going to end the article that abruptly.

Conclusion

And there go the 3 different discussions about the specifics of modern Rx in commercial programming!

About Me

Hi, I’m Isaac, I’ve been doing a bit of full-time commercial iOS programming for nearly 3 years and I have pledged to participate in the No Writing-a-shitty-article-on-medium November. As you can see, unsuccessfully.

Hey, if you liked this article 🤔
1. please clap once 🙏
2. follow me on GitHub 🥴
3. check out my website 👬
4. my wife left me

Flawless iOS

This article is published at Flawless iOS. Thank you very much for reviewing my boring ass entries!

Here are my top reads that are currently in rotation:

1. Fighting state redundancy in MVVM by Alexey Naumov.

I found this one a great illustration why we envy callback-based event propagation with multiple accepted parameters. But what’s even more important is that he shows you how to write reactive interface for your views, although making the service calls reactive and getting rid of all the arguably unnecessary BehaviorRelay’s wouldn’t go a miss. If you’s a starter — check this one out.

2. Achieving maximum test readability at no cost for iOS by Victor Magalhães.

It’s not like you and I have worked in many companies where the business is too fond of automated unit/ui testing. But to see a mocha-esque approach for declaring unit tests and conditions is actually a lot of fun, I ain’t even knew this was a thing in XCTest.