HTTP uses TCP, so naturally this is going to be built on top of the previous post.
To goal is to receive a HTTP request and respond with 200 OK where the body has information about the request received.
A HTTP request has three parts:
Example:
GET /index.html HTTP/1.1Host: marcelofernandes.dev
{:hello "There"}
Convenient copy of the previous TCP server that we’re going to built on top of.
(ns user (:require [clojure.string :as str]) (:import (java.net ServerSocket) (java.io BufferedReader InputStreamReader PrintWriter)))
(defn- handle-input [{:keys [in out handler]}] (loop [] (when-let [line (.readLine in)] (println "Received:" line) (.println out (handler line)) (recur))))
(defn- client-handler [client-socket handler] (future (try (with-open [in (BufferedReader. (InputStreamReader. (.getInputStream client-socket))) out (PrintWriter. (.getOutputStream client-socket) true)] (handle-input {:in in :out out :handler handler})) (catch Exception e (println "Client error:" (.getMessage e))) (finally (when-not (.isClosed client-socket) (.close client-socket) (println "Client disconnected by error"))))))
(defn create-tcp-server [{:keys [port handler]}] (let [state (atom {:server-socket nil :clients []}) start-server (fn [] (when (nil? (:server-socket @state)) (reset! state {:server-socket (ServerSocket. port) :clients []}) (println "Server started on port" port) (future (try (loop [] (let [client-socket (.accept (:server-socket @state))] (swap! state update :clients conj client-socket) (client-handler client-socket handler) (recur))) (catch Exception e (println "Server error:" (.getMessage e))) (finally (when-not (.isClosed (:server-socket @state)) (.close (:server-socket @state)) (println "Server stopped by error"))))))) stop-server (fn [] (when-let [server-socket (:server-socket @state)] (doseq [client-socket (:clients @state)] (.close client-socket) (println "Client disconnected by server")) (.close server-socket) (println "Server stopped by user") (reset! state {:server-socket nil :clients []})))] {:start start-server :stop stop-server}))
(let [{:keys [start stop]} (create-tcp-server {:port 1230 :handler str/upper-case})] (start) (Thread/sleep 10000) (stop))
To start we can test the behavior of this server by making a request with curl
:
curl localhost:1230
The logs are:
Server started on port 1230Received: GET / HTTP/1.1Received: Host: localhost:1230Received: User-Agent: curl/8.4.0Received: Accept: */*Received:Client error: Connection resetClient disconnected by serverServer stopped by userServer error: Socket closed
We can see that it reads one line at a time, first getting the request line, then the headers, followed by an empty line. That doesn’t say anything about the body. Let’s try with a POST:
curl --data '{:hello "There"}' --header 'Content-Type: application/edn' localhost:1230
The logs:
Server started on port 1230Received: POST / HTTP/1.1Received: Host: localhost:1230Received: User-Agent: curl/8.4.0Received: Accept: */*Received: Content-Type: application/ednReceived: Content-Length: 16Received:Client error: Connection resetClient disconnected by serverServer stopped by userServer error: Socket closed
We got some extra headers but we’re not able to see the {:hello "There"}
yet.
I want to transform that request into a data structure that I can work with. For the request above that would be:
{:method :post :path "/" :headers {"Host" "localhost:1230" "User-Agent" "curl/8.4.0" "Accept" "*/*" "Content-Type" "application/edn" "Content-Length" "16"}} :body {:hello "There"}
The handle-input
doesn’t need a handler
function anymore. To keep the REPL flow going we can structure the code like this:
(ns user (:require [clojure.string :as str]) (:import (java.net ServerSocket) (java.io BufferedReader InputStreamReader PrintWriter)))
(defn- handle-input [{:keys [in _out]}] (loop [request {} state :idle] (cond (= state :idle) (if-let [line (.readLine in)] (let [[method path protocol] (str/split line #"\s+")] (recur (assoc request :method (-> method str/lower-case keyword) :path path :protocol protocol) :headers)) (println "Client disconnected"))
(= state :headers) (do (println request) (println "Still need to parse headers")))))
(defn- client-handler [client-socket] (future (try (with-open [in (BufferedReader. (InputStreamReader. (.getInputStream client-socket))) out (PrintWriter. (.getOutputStream client-socket) true)] (handle-input {:in in :out out})) (catch Exception e (println "Client error:" (.getMessage e))) (finally (when-not (.isClosed client-socket) (.close client-socket) (println "Client disconnected by error"))))))
(defn create-tcp-server [{:keys [port]}] (let [state (atom {:server-socket nil :clients []}) start-server (fn [] (when (nil? (:server-socket @state)) (reset! state {:server-socket (ServerSocket. port) :clients []}) (println "Server started on port" port) (future (try (loop [] (let [client-socket (.accept (:server-socket @state))] (swap! state update :clients conj client-socket) (client-handler client-socket) (recur))) (catch Exception e (println "Server error:" (.getMessage e))) (finally (when-not (.isClosed (:server-socket @state)) (.close (:server-socket @state)) (println "Server stopped by error"))))))) stop-server (fn [] (when-let [server-socket (:server-socket @state)] (doseq [client-socket (:clients @state)] (.close client-socket) (println "Client disconnected by server")) (.close server-socket) (println "Server stopped by user") (reset! state {:server-socket nil :clients []})))] {:start start-server :stop stop-server}))
(comment (def server (create-tcp-server {:port 1230}))
((:start server))) ;; to start
((:stop server)) ;; to stop
Then we can just evaluate handle-input
any time there is a change. When receiving a new request it will use the new version of the function.
When sending a request like this:
curl --data '{:hello "There"}' --header 'Content-Type: application/edn' localhost:1230
the logs show:
{:method :post, :path /, :protocol HTTP/1.1}Still need to parse headers
Parsing the headers is a matter of reading line by line and splitting by :
.
(defn- handle-input [{:keys [in _out]}] (loop [request {} state :idle] (condp = state :idle (if-let [line (.readLine in)] (let [[method path protocol] (str/split line #"\s+")] (recur (assoc request :method (-> method str/lower-case keyword) :path path :protocol protocol) :headers)) (println "Client disconnected"))
:headers (if-let [line (.readLine in)] (if (empty? line) (recur request :done) (let [header-vector (str/split line #": ") request-with-new-header (assoc-in request [:headers (first header-vector)] (second header-vector))] (recur request-with-new-header :headers))) (recur request :done))
:done (do (println request) (println "Done parsing the request")))))
which logs
{:method :post, :path /, :protocol HTTP/1.1, :headers {Host localhost:1230, User-Agent curl/8.4.0, Accept */*, Content-Type application/edn, Content-Length 16}}Done parsing the request
To write the response back we just need to answer according to HTTP protocol:
HTTP/1.1 200 OK{headers here}
{body here}
In our case we won’t sent any headers and just send our structure as the body.
(defn- handle-input [{:keys [in out]}] (loop [request {} state :idle] (condp = state :idle (if-let [line (.readLine in)] (let [[method path protocol] (str/split line #"\s+")] (recur (assoc request :method (-> method str/lower-case keyword) :path path :protocol protocol) :headers)) (println "Client disconnected"))
:headers (if-let [line (.readLine in)] (if (empty? line) (recur request :done) (let [header-vector (str/split line #": ") request-with-new-header (assoc-in request [:headers (first header-vector)] (second header-vector))] (recur request-with-new-header :headers))) (recur request :done))
:done (do (.println out "HTTP/1.1 200 OK\r\n\r\n") (.println out (str request)) (println "Done parsing the request")))))
When I open the browser and navigate to http://localhost:1230 I see this response (unformatted):
{:method :get :path "/" :protocol "HTTP/1.1" :headers {"Sec-Fetch-Dest" "document" "Sec-Fetch-Mode" "navigate" "Upgrade-Insecure-Requests" "1" "Sec-Fetch-User" "?1" "User-Agent" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:128.0) Gecko/20100101 Firefox/128.0" "Priority" "u=0, i" "Sec-Fetch-Site" "none" "Accept-Encoding" "gzip, deflate, br, zstd" "Accept-Language" "en-US,en;q=0.5" "Connection" "keep-alive" "Accept" "text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,image/png,image/svg+xml,*/*;q=0.8" "Host" "localhost:1230"}}
The body is a little trickier because we can’t just use readLine
. It doesn’t have a line feed (\n) or carriage return (\r) or anything like that. Instead in this basic attempt we rely on Content-Length header.
This request:
curl --data '{:hello "There"}' --header 'Content-Type: application/edn' localhost:1230
Has a content length of 16.
Then we can check if the content length header is present, make a huge assumption that it’s telling the truth, and read that many characters:
(defn- handle-input [{:keys [in out]}] (loop [request {} state :idle] (condp = state :idle (if-let [line (.readLine in)] (let [[method path protocol] (str/split line #"\s+")] (recur (assoc request :method (-> method str/lower-case keyword) :path path :protocol protocol) :headers)) (println "Client disconnected"))
:headers (if-let [line (.readLine in)] (if (empty? line) (recur request :body) (let [[key value] (str/split line #": ")] (recur (assoc-in request [:headers key] value) :headers))) (recur request :body))
:body (let [content-length (get-in request [:headers "Content-Length"])] (if-not content-length (recur request :done) (let [size (parse-long content-length) body (char-array size)] (doseq [i (range size)] (aset body i (char (.read in)))) (recur (assoc request :body (String. body)) :done))))
:done (do (.println out "HTTP/1.1 200 OK\r\n\r\n") (.println out (str request)) (println "Done parsing the request")))))
The response for the previous request is:
{:method :post :path "/" :protocol "HTTP/1.1" :headers {"Host" "localhost:1230" "User-Agent" "curl/8.4.0" "Accept" "*/*" "Content-Type" "application/edn" "Content-Length" "16"} :body "{:hello \"There\"}"}
The complete code is this:
(ns user (:require [clojure.string :as str]) (:import (java.net ServerSocket) (java.io BufferedReader InputStreamReader PrintWriter)))
(defn- handle-input [{:keys [in out]}] (loop [request {} state :idle] (condp = state :idle (if-let [line (.readLine in)] (let [[method path protocol] (str/split line #"\s+")] (recur (assoc request :method (-> method str/lower-case keyword) :path path :protocol protocol) :headers)) (println "Client disconnected"))
:headers (if-let [line (.readLine in)] (if (empty? line) (recur request :body) (let [[key value] (str/split line #": ")] (recur (assoc-in request [:headers key] value) :headers))) (recur request :body))
:body (let [content-length (get-in request [:headers "Content-Length"])] (if-not content-length (recur request :done) (let [size (parse-long content-length) body (char-array size)] (doseq [i (range size)] (aset body i (char (.read in)))) (recur (assoc request :body (String. body)) :done))))
:done (do (.println out "HTTP/1.1 200 OK\r\n\r\n") (.println out (str request)) (println "Done parsing the request")))))
(defn- client-handler [client-socket] (future (try (with-open [in (BufferedReader. (InputStreamReader. (.getInputStream client-socket))) out (PrintWriter. (.getOutputStream client-socket) true)] (handle-input {:in in :out out})) (catch Exception e (println "Client error:" (.getMessage e))) (finally (when-not (.isClosed client-socket) (.close client-socket) (println "Client disconnected by error"))))))
(defn create-tcp-server [{:keys [port]}] (let [state (atom {:server-socket nil :clients []}) start-server (fn [] (when (nil? (:server-socket @state)) (reset! state {:server-socket (ServerSocket. port) :clients []}) (println "Server started on port" port) (future (try (loop [] (let [client-socket (.accept (:server-socket @state))] (swap! state update :clients conj client-socket) (client-handler client-socket) (recur))) (catch Exception e (println "Server error:" (.getMessage e))) (finally (when-not (.isClosed (:server-socket @state)) (.close (:server-socket @state)) (println "Server stopped by error"))))))) stop-server (fn [] (when-let [server-socket (:server-socket @state)] (doseq [client-socket (:clients @state)] (.close client-socket) (println "Client disconnected by server")) (.close server-socket) (println "Server stopped by user") (reset! state {:server-socket nil :clients []})))] {:start start-server :stop stop-server}))
(comment (def server (create-tcp-server {:port 1230}))
((:start server)) ((:stop server)))
It’s full of problems and security issues. But it’s a nice start for creating a basic HTTP server using only the Socket interface.