Welcome to Software Development on Codidact!
Will you help us build our independent community of developers helping developers? We're small and trying to grow. We welcome questions about all aspects of software development, from design to code to QA and more. Got questions? Got answers? Got code you'd like someone to review? Please join us.
Using an existing web server vs writing your own
When writing a dynamic web service, you broadly speaking have two paths:
-
Use an existing web server (e.g. Apache, Nginx or Lighttpd) to handle the "raw" web requests and implement your own code as a separate process that communicates with the server using a gateway protocol (e.g. FastCGI). A typical PHP service is a great example of this.
-
Skip the general purpose web server altogether and write your program to handle the requests directly. Typically you would use some web framework which takes care of most technicalities. Services written in Rust tend to take this path.
My question is that if you are using a language that is generally suitable for implementing a standalone web server (such as Rust), is there any advantage with the first path? General pros and cons of both architectures are also welcome.
4 answers
You are accessing this answer with a direct link, so it's being shown above all other answers regardless of its score. You can return to the normal view.
3rd Option: You put a web server like Nginx, etc in front of your application server which can also handle 'raw' network traffic. Even in the PHP scenario this is common.
There are a few reason I do this:
- Something like Nginx is more tested than your framework's internal networking code. So it's safer to be exposed to the internet. Its scope is more limited (ie it only does networking) which means there's less to be exploited. This security applies strictly to the handling of network requests, obviously your application can still be exploited in other ways even behind a proxy.
- There are more 'network transport' level features that are going to be available because that's all those those products are designed to do.
- There are some tasks that don't make sense to include in your application code at all. For example load balancing between several copies of your application server.
- You can hand off your application server to a devops/sysadmin and let them manage the network traffic. If the application is designed to solely handle its own traffic, those teammates will now need to know how to code in your language to configure it.
- Simplifies your application server code. For example, handling HTTPS can require extra configuration specific to your framework. You'll still require that configuration if you put an existing web server in front of you application server but there is likely to be much more resources available for figuring out how to do that.
I'm not familiar with Rust, but in ASP.net there's Kestrel which is a webserver built into the framework. But typically you would not deploy it facing the internet.
Their reasoning (reverse proxy being the 'existing web server'):
Even if a reverse proxy server isn't required, using a reverse proxy server might be a good choice. A reverse proxy:
- Can limit the exposed public surface area of the apps that it hosts.
- Provides an additional layer of configuration and defense.
- Might integrate better with existing infrastructure.
- Simplifies load balancing and secure communication (HTTPS) configuration. Only the reverse proxy server requires the X.509 certificate for the public domain(s). That server can communicate with the app's servers on the internal network using plain HTTP or HTTPS with locally managed certificates. Internal HTTPS increases security but adds significant overhead.
The only case where I wouldn't care to put a proxy in front of my application server would be for proofs of concept and application servers that don't face the internet.
1 comment thread
The reverse proxy is frankly a "best practice" architecture, not just for security but also for resource limiting of your hard-working application server.
Often the reverse proxy will communicate using web protocols anyway, so you are reusing webserver comms for internal comms, which will often make the whole thing easier to implement.
Most programming languages have webserver services or libraries available, its not really about using a dedicated webserver or using your app as a webserver, the reason is architectural. Forget the networking protocol, its about that security, rate limiting, load balancing and every other good thing you get from this architecture.
Note that this is already happening when you "directly" connect a webserver to the web, it'll be behind all manner of routers, CDNs and NAT bridges to get the data to you.
0 comment threads
A related approach is to extend or customize an existing open source web server or web library.
If you can code in C or C++ (on Linux), consider writing your web server software above libonion (it is coded mostly in C) or Wt (mostly in C++) libraries (both are developed in the European union). Another approach would involve extending lighttpd perhaps with your plugin coded in C. Using libcurl the same code can be both a Web server and a web client (e.g. for proxy, caching, or running in a DMZ at the border of a corporate internal network). Your C++ code might also use VMime for complex email processing or sending.
If you can code or prefer to code in Ocaml, consider using the Ocsigen framework.
Of course, you could still code using PHP.
Of course, most web servers are extensible using the old CGI protocol (which I don't recommend, since it is starting a process for most HTTP requests) or the FastCGI one.
Above all, you need to understand how critical and reliable should be your web application, and who are the intended users. For example, the web interface to a hospital is more critical than most small web e-commerce sites. Some web interfaces are accessed from laptops and desktops, others should be usable from mobile phones or small tablets.
Be aware of the C10K problem, and of web accessibility and internationalization issues, and of ICAP. It may (or not) be relevant to your project.
The advantage of using an existing Web server (communicating with FastCGI or HTTP to another server) is that the static contents of your web site (photos, texts) are still working even if your customization is crashing.
NB. My pet open source project is RefPerSys (an inference engine, GPLv3+ licensed, which is a web client).
0 comment threads
Is this project a web server?
Or a web application?
The people who have made nginx, apache, etc.. Are building web servers. 100% of their effort on those projects goes towards making the web server faster and safer. And they are huge built by massive communities. Are widely used so have a lot of in the wild proof of their security.
A web server that is built as part of building a web application will not receive 100% of the effort... It probably won't receive 10% of the effort. It will not have 1000s of people who have worked on it. There ill not be 1000s of instances of it out in the wild being hacked, and exploited, teaching the developers what security flaws existed in their original design.
A web server team will have dedicated time, and energy to finding and fixing edge cases that a web application team that roles their own web server doesn't have the time, man power, or knowledge to even dream of.
TLDR: unless the goal is to build a web server. Don't build a web server. Build the web application, or else by the time the web server is built, the time for your new cutting edge web application will be in the distant past.
0 comment threads