Self-contained systems [SCS] provide a way to build systems that each have their own web UI and to make them look like one system from the user’s perspective. This means that some way of integrating these systems is necessary. A lot has already been written and said by the great minds of IT about the points of integration within the frontend (REST, messaging, data replication, compile-time, …). At the moment, I think that the question of how to integrate systems in the web frontend is a bit more exciting, although it is discussed much less frequently.
Here, too, there are probably zillions of different approaches. In this article, I want to examine one particular approach that may be the most problematic for people: Transclusion of HTML from other systems. Conceptionally, this is about replacing a link with the content of the <body> tag that you will get if you make a HTTP GET request with an Accept: text/html head to that link.
Of course, it’s not yet clear where exactly this substitution takes place. Once more, there are multiple variants that I would like to examine by means of a small example.
Technical options for transclusion
Let’s imagine a web portal of a (modern 😉) airline: If one of its systems is responsible for the status of flights (delays etc.), and another one is responsible for the individual bookings, it may be reasonable to show the status of the respective flight directly in the bookings overview. If this integration in the frontend is supposed to happen by means of transclusion there are at least two very different ways of implementing this:
- Transclusion in a web server or reverse proxy
- Transclusion in the browser via Ajax
For the first of these approaches, Edge Side Includes [ESI] or Server Side Includes [SSI] are viable options. Here, the page in which the HTML is going to be embedded is searched for appropriate placeholders (for example <esi:include src='http://mein.server.de/foo/bar'>) in a web server or reverse proxy. For each of these placeholders, a URI is extracted and a HTTP GET request is executed to that URI, ultimately replacing the placeholder in the embedding page with the content of the respective HTTP response.
Of course, the crucial point of these techniques is the reaction speed of the backend. For instance, if the bookings overview renders a list of bookings that contains one <esi:include> beneath each booking pointing to the respective flight status, a reverse proxy first needs to replace each of those <esi:include> tags before the response can even be sent to the client. Now, if the flight status system responds slowly for only a single one of these requests, the whole response will get delayed and the user will not see any reaction for quite a while.
If you approached this problem on the client side, a few lines of jQuery code would be enough (at least for a prototype):
$('a.replace-link').each(function() {
var link = $(this);
var content = $('<div></div>').load(link.attr('href'), function() {
link.replaceWith(content);
});
});Of course, this only works as long as the link target meets the requirements of the same origin policy [SOP]. On the other hand, this approach has the nice property of processing all transclusions on a page independently from one another and asynchronously, so that errors will only have a local effect. Moreover, it is relatively easy to transclude only a specific subtree of the embedded document instead of always transcluding the whole content of the <body tag (see [JQL], for example).
Which of the two alternatives is better can be determined on a case-by-case basis. Generally, server-side transclusion seems to be reasonable for the primary content of a page. However, having to transclude primary content into a page feels like a “smell” to me. After all, an SCS should actually contain the core of its functionality instead of having to transclude it.
Personally, I would generally prefer the second way of doing transclusion. Things that typically cannot be covered using this appproach are main menus or footers. I would tend to integrate these at compile time as static assets and would focus on being able to deploy quickly and often. This is fine because changes to these assets are usually
- rare
- often not critical
- easier to switch per system with global feature toggles instead of global magic.
Styling and functionality
Both approaches always have one common problem: If you simply add arbitrary new content to the DOM – regardless of the mechanism – you also need to make sure that this content will be displayed decently (styling) and that it will work as intended (particularly with regard to JavaScript). At the end of the day, it is about the question of which static assets (CSS and JavaScript) get pulled in and where they are coming from.
And, who would have thought, once again we are faced with multiple options.
Common assets
The approach that seems to be easiest is probably to restrict transclusion to elements from a central assets repository. For example, if our flight management were based on Bootstrap [BS], there could be a rule that transcluded content may exclusively contain elements from Bootstrap and may not contain any custom styling.
Each transcluding view (in our example, the bookings overview) has to serve Bootstrap so that the embedded content will be displayed neatly.
Custom assets
If that sounds plausible to you, you don’t have to read on and can stop here. To me, it sounds somewhat unrealistic that Bootstrap will be enough to implement the UX/UI requirements. Hence, we will likely not get around maintaining the core assets ourselves. In our example, there would be a third special system that is responsible for providing the core styles and scripts. Transcluding content will then have to integrate this accordingly.
One danger of this approach is that this assets project will likely not exist yet at the beginning of the overall project. This means that it has to be developed in parallel to the other systems and will therefore violate a lot of the principles of self-contained systems (just not in the backend, but in the frontend). However, since some papers and strange architecture manifestos are quite tolerant, I have already seen this approach in the wild a couple of times. The successful cases usually have a very good assets team, consisting of members of each of the teams responsible for the other systems. If you take this problem lightly, however, the result is certainly endangering the success of the project.
Versioning
One property that is especially true for well-maintained custom assets is that they are frequently changing. Of course, this is can be the case for the Bootstrap case described before, if you, say, want to update Bootstrap from v3 to v4. Hence, if the booking overview in our example is already serving Bootstrap 4 while the flight status is still relying on the presence of Bootstrap 3, the flight status will probably not be displayed correctly in the embedding page and we have a problem.
The only thing that helps in that case is to update the transcluded content before the embedding views and to provide both versions for a while. In our example, there would be two versions of the flight status: one that is compatible with Bootstrap 3 and one that is compatible with Boostrap 4. Which version gets embedded could be specified either by the URI or by means of HTTP headers.
Oh well… that sounds like work (which we have to do for the backend APIs all the same).
Self-contained assets
If you take SCS architecture literally, the assets that are relevant to transclusion would have to be a part of the respective system just like the content itself. In our example, the bookings overview would have its own styles in order to display its content correctly. In addition to that, we somehow need to make sure that the styles and scripts of the flight status system will be present in the current window.
At first, that sounds obvious, especially when you pretend that styles and scripts are isolated. Unfortunately they aren’t. However, if you manually provide for the highest possible isolation, for example by preventing collisions of CSS selectors (e.g. using system-specific HTML class prefixes), you can come to grips with this problem.
One question remains, though: How do the styles and scripts get into the transcluding system?
Inline
The simplest approach would probably be to deliver <style> and <script> tags together with the content. True inline styles (<div style="...">) are conceivable as well. Since the latter has a massive effect on the specificity of CSS selectors, it’s not really an option.
In our example, this approach would be disadvantageous as soon as the bookings overview contains more than one booking. In that case, the same styles and scripts would be delivered multiple times. Nevertheless, to me this approach seems to be more frowned upon at the moment than it deserves.
Linking of required styles and scripts
If every system simply provided its own styles and scripts, transcluding systems would merely have to include those. In our example, the bookings overview would provide its own assets and, in addition to that, would include the styles from the flight status system that are relevant for transclusion using a <link rel="stylesheet" href="..."> in the <head> (scripts would be handled in the same way).
One disadvantage of this approach becomes apparent if a system embeds many other systems. In such a situation, the browser would execute quite a few requests in order to load the other systems' assets, which are often not acceptable (especially without HTTP 2). Moreover, the question in such a scenario is how to include the assets with a version in the URL, since you typically want to provide assets with a very high value for the Cache-Control: max-age directive. So how does the bookings overview know which version of the assets of the flight status system to include?
This problem needs to be addressed on a different level (my personal preference: JSON Home [JSH]).
Artificial centralization
Another alternative would be to collect those styles and scripts that are needed for transclusion from multiple systems during the build and put them into a common styles and scripts bundle. This approach mainly addresses the problem of the previously described technique (linking). However, it is probably too big for many projects due to a lack of standard products.
Jumble
Of course, both approaches (central assets and self-contained assets) can be combined. For example, you could have a central (relatively stable) framework like Bootstrap or a previosly built custom framework, and transcluding systems could additionally serve their own customizations on top of that.
Once the project enters a stable phase or you notice that more than one system requires the same assets, those assets could be consolidated in the central assets one after another.
What now?
If I were to implement the example I discussed, I would tend to use a mostly Ajax-based approach to transclusion. I would probably have an assets team (perhaps consisting of members of each system team) build the assets, but still allow linking to system-specific assets. The first task of the central assets team would probably be to provide the styling of the common page frame, the transclusion logic, and some important components (buttons, links, text, …).
Is there a moral?
What I notice over and over again is that you often recreate iFrames. These have a high isolation, can potentially interact [SOP] and you hardly have to do anything for that. However, they are frowned upon, which is likely mainly due to the fact that the embedding page determines the size (particularly the height) of the embedded window without actually being able to know what they should be. Nevertheless, I think it’s good idea to keep this alternative in mind, so that at least sometimes you will be able to avoid the complex subject of transclusion.
This article was originally published in German and translated by Daniel Westheide.