1 2 Previous Next 26 Replies Latest reply on Nov 19, 2008 2:26 AM by trustin

    Netty HTTP transport

    ataylor

      A starter for 10.

      I guess the first thing we need is a Http Codec. I guess the codec just needs to decipher the headers and then the buffer content will be held in the body. Once we have the codec then we can build an engine that takes the http requests and translates into Netty Buffers.

      As mentioned on IRC we probably need to support the client both polling for responses and keeping the connection open but i guess it makes sense to concentrate on one or the other first.





        • 1. Re: Netty HTTP transport
          trustin

          Good start. To write a HTTP codec, we need message classes that represents HTTP requests and responses. It will resemble that of Servlets, but it should be more user-friendly than that.

          Actually I have a draft implementation of the message model in my local copy. Let me check it in soon and ping here.

          • 2. Re: Netty HTTP transport
            ataylor

            How did you envisage this working from the client side, were you just going to build on top of URLConnection or one of the Http client API's or write something from scratch?

            • 3. Re: Netty HTTP transport
              trustin

              Bad news: My local copy has gone. I can't find it anymore from my disk. *sigh*

              Here's what I have been doing to implement HTTP on top of Netty.

              1) A codec is basically an implementation of the ChannelHandler interface. Therefore, HTTP codec classes should be placed in the org.jboss.netty.handler.codec.http package.

              2) We should have at least three types: HttpRequest, HttpResponse and HttpMethod. HttpMethod should be an enum of all available HTTP methods like GET, POST, etc.

              HttpRequest and HttpResponse will represent an actual HTTP request and response respectively. They will provide essential getters that are required to implement an HTTP server and client. (We could add some convenience access methods like cookie accessors, but that's another story.)

              According to the HTTP spec, HTTP request and respose share a couple properties such as header, and therefore it would be a better idea to start from a super interface whose name is 'HttpMessage' IMHO.

              3) There's no need to implement HTTP codec to support large chunk of data directly, which means a HttpRequest.getContent() can just return a complete ChannelBuffer instead of a special stream type.

              However, Chunked encoding should be supported instead to enable COMET, which will eventually allow us to use HTTP as a connection-oriented socket. (Bill Burke is also interested in this.)

              Supporting chunked encoding is an interesting problem though because the codec should generate additional messages to encode or decode the chunk which is not first. Probably we will need a new type to support chunked encoding, such as 'HttpChunk' and let the user (higher level HTTP client/server implementor) maintain the state by oneself.

              Once, this codec is ready, I think implementing a highly customized HTTP server shouldn't be difficult at all. WDYT?

              • 4. Re: Netty HTTP transport
                trustin

                 

                "ataylor" wrote:
                How did you envisage this working from the client side, were you just going to build on top of URLConnection or one of the Http client API's or write something from scratch?


                The plan is to write a codec for both client and server side, which means we need two encoders and two decoders - HttpRequestEncoder, HttpResponseEncoder, HttpRequestDecoder and HttpResponseDecoder. So, it's writing something from scratch.

                Implementation of an encoder is pretty straighforward, but it's not for decoders. I think we can use ReplayingDecoder to minimize the decoder implementation time.

                • 5. Re: Netty HTTP transport
                  ataylor

                   

                  1) A codec is basically an implementation of the ChannelHandler interface. Therefore, HTTP codec classes should be placed in the org.jboss.netty.handler.codec.http package.


                  makes sense

                  2) We should have at least three types: HttpRequest, HttpResponse and HttpMethod. HttpMethod should be an enum of all available HTTP methods like GET, POST, etc.

                  HttpRequest and HttpResponse will represent an actual HTTP request and response respectively. They will provide essential getters that are required to implement an HTTP server and client. (We could add some convenience access methods like cookie accessors, but that's another story.)

                  According to the HTTP spec, HTTP request and respose share a couple properties such as header, and therefore it would be a better idea to start from a super interface whose name is 'HttpMessage' IMHO.


                  Yes i agree the basic default message shouuld contain things like header, protocol version etc etc.

                  3) There's no need to implement HTTP codec to support large chunk of data directly, which means a HttpRequest.getContent() can just return a complete ChannelBuffer instead of a special stream type.


                  Then we just pass on the buffer in the usual fashion.

                  However, Chunked encoding should be supported instead to enable COMET, which will eventually allow us to use HTTP as a connection-oriented socket. (Bill Burke is also interested in this.)


                  Do we need to support this for the first implementation?

                  Once, this codec is ready, I think implementing a highly customized HTTP server shouldn't be difficult at all. WDYT?


                  yes, I think if we concentrate on getting the message model complete we can go from there.

                  • 6. Re: Netty HTTP transport
                    ataylor

                     

                    The plan is to write a codec for both client and server side, which means we need two encoders and two decoders - HttpRequestEncoder, HttpResponseEncoder, HttpRequestDecoder and HttpResponseDecoder. So, it's writing something from scratch.


                    I guess the encoder just writes the headers followed by the body but with thge decoder we need towait for the whole of the body to be delivered before translating the content into a nettybuffer?

                    • 7. Re: Netty HTTP transport
                      trustin

                       

                      "ataylor" wrote:
                      The plan is to write a codec for both client and server side, which means we need two encoders and two decoders - HttpRequestEncoder, HttpResponseEncoder, HttpRequestDecoder and HttpResponseDecoder. So, it's writing something from scratch.


                      I guess the encoder just writes the headers followed by the body but with thge decoder we need towait for the whole of the body to be delivered before translating the content into a nettybuffer?


                      Yes. That's why implementing an encoder is relatively easier than implementing a decoder.

                      Netty provides a couple ways to write a decoder easily. For example, take a look at this: http://tinyurl.com/57dtcb

                      ReplayingDecoder simplifies it even further. It allows you to write a decoder just like writing a blocking decoder: http://tinyurl.com/5tkwnw



                      • 8. Re: Netty HTTP transport
                        ataylor

                        I could probably look to get the message model and the encoder/decoder done for next week. Then we can discuss the next step.

                        I'll create a branch to work on.

                        • 9. Re: Netty HTTP transport
                          trustin

                           

                          "trustin" wrote:
                          ReplayingDecoder simplifies it even further. It allows you to write a decoder just like writing a blocking decoder: http://tinyurl.com/5tkwnw

                          Forgot to add a link. :)

                          • 10. Re: Netty HTTP transport
                            trustin

                             

                            "ataylor" wrote:
                            I could probably look to get the message model and the encoder/decoder done for next week. Then we can discuss the next step.

                            I'll create a branch to work on.

                            OK. Please let me know your SVN username. I will add you to the developer group.

                            • 11. Re: Netty HTTP transport
                              trustin

                               

                              "ataylor" wrote:
                              "trustin" wrote:
                              3) There's no need to implement HTTP codec to support large chunk of data directly, which means a HttpRequest.getContent() can just return a complete ChannelBuffer instead of a special stream type.


                              Then we just pass on the buffer in the usual fashion.


                              Yes.

                              "ataylor" wrote:
                              "trustin" wrote:
                              However, Chunked encoding should be supported instead to enable COMET, which will eventually allow us to use HTTP as a connection-oriented socket. (Bill Burke is also interested in this.)


                              Do we need to support this for the first implementation?


                              I don't think so. It can be done after the basic implementation is done.

                              "ataylor" wrote:
                              "trustin" wrote:
                              Once, this codec is ready, I think implementing a highly customized HTTP server shouldn't be difficult at all. WDYT?


                              yes, I think if we concentrate on getting the message model complete we can go from there.


                              Cool. :)

                              • 12. Re: Netty HTTP transport
                                ataylor
                                • 13. Re: Netty HTTP transport
                                  ataylor

                                  Ive got the main mechanics of the message model and codecs done. Ive also added a couple of examples, one that sends a request to the jboss netty page and a client/server example that sends a basic string message. There are lots of bells and whistles still to be added but i thought for JBM 2.0 we probably only need the basic implementation for now. Also, I'm not yet dealing with keep-alive vs close after a request.

                                  Trustin, can you review what ive done so far and then we can discuss what is needed next.

                                  • 14. Re: Netty HTTP transport
                                    ataylor

                                    Also I wasn't sure on you're testing strategy so I've yet to add any tests. Let me know what kind of tests you prefer and write some.

                                    1 2 Previous Next