I agree with the basic thesis here, but I feel like we could use a few more good examples. The Rust example is evocative, and certainly language implementations do a lot of heavy lifting of various forms to make it possible for people to write less complex code. Certainly, type systems are a nice example of that.
Maybe another good example is ML systems like Tensorflow and PyTorch.
What about networks? Networking infrastructure is in some ways aggressively simple, where the abstractions that are provided are very limited. But...networks grow in complexity to continue to provide these simple abstractions even in circumstances where it's not so simple to do so. Like subnets, which used to correspond to concrete pieces of hardware, but are now provided virtually.
Anyway, I find it interesting despite the fact that the thesis here resonates pretty strongly with me, I'm having a hard time coming up with a long list of good examples. The examples that do come to mind are mostly terribly specific to this or that application I've worked on.
I think ML systems fit well into this as well. (As you know, these ML systems share a lot of properties with high level programming languages, and provide similar facilities.)
Networking is an interesting one. On the one hand, the amount of sophistication that is required to provide a simple abstraction can oftentimes be staggering. On the other hand, networks are also not reliable, and we lean heavily on protocols like TCP to provide end-to-end reliability -- we solve the "hard problems" further up the stack. It's interesting to think about how this argument gels (or doesn't) with the "end to end" argument: https://web.mit.edu/Saltzer/www/publications/endtoend/endtoend.pdf.
Unfortunately, I think this article misses the point. The example seems to reinforce the main idea of the article but that's only because it's a single example of a small part of actual production systems. My understanding of your statement "we should prefer concentrated complexity to diffuse complexity" is that we should tend towards monoliths, multipurpose systems.
The danger here is that concentrating complexity in a single (or few) component(s) isn't desirable and will eventually lead to that component failing or becoming unmaintainable. However, it's true that your system's complexity should not be randomly spread across it.
I think the correct idea is to localize related complexities into single components (or potentially well-defined groups of components). Although you could use an SQL database which has encapsulated all the complexity into a single dependency, you can just as easily use a distributed storage system with eventual consistency and abstract it away from the rest of your application using a single component.
This is akin the Unix philosophy, where you should have simple components with well-defined functionalities.
I agree with the basic thesis here, but I feel like we could use a few more good examples. The Rust example is evocative, and certainly language implementations do a lot of heavy lifting of various forms to make it possible for people to write less complex code. Certainly, type systems are a nice example of that.
Maybe another good example is ML systems like Tensorflow and PyTorch.
What about networks? Networking infrastructure is in some ways aggressively simple, where the abstractions that are provided are very limited. But...networks grow in complexity to continue to provide these simple abstractions even in circumstances where it's not so simple to do so. Like subnets, which used to correspond to concrete pieces of hardware, but are now provided virtually.
Anyway, I find it interesting despite the fact that the thesis here resonates pretty strongly with me, I'm having a hard time coming up with a long list of good examples. The examples that do come to mind are mostly terribly specific to this or that application I've worked on.
I think ML systems fit well into this as well. (As you know, these ML systems share a lot of properties with high level programming languages, and provide similar facilities.)
Networking is an interesting one. On the one hand, the amount of sophistication that is required to provide a simple abstraction can oftentimes be staggering. On the other hand, networks are also not reliable, and we lean heavily on protocols like TCP to provide end-to-end reliability -- we solve the "hard problems" further up the stack. It's interesting to think about how this argument gels (or doesn't) with the "end to end" argument: https://web.mit.edu/Saltzer/www/publications/endtoend/endtoend.pdf.
Unfortunately, I think this article misses the point. The example seems to reinforce the main idea of the article but that's only because it's a single example of a small part of actual production systems. My understanding of your statement "we should prefer concentrated complexity to diffuse complexity" is that we should tend towards monoliths, multipurpose systems.
The danger here is that concentrating complexity in a single (or few) component(s) isn't desirable and will eventually lead to that component failing or becoming unmaintainable. However, it's true that your system's complexity should not be randomly spread across it.
I think the correct idea is to localize related complexities into single components (or potentially well-defined groups of components). Although you could use an SQL database which has encapsulated all the complexity into a single dependency, you can just as easily use a distributed storage system with eventual consistency and abstract it away from the rest of your application using a single component.
This is akin the Unix philosophy, where you should have simple components with well-defined functionalities.