{"id":2219,"date":"2008-07-30T14:54:34","date_gmt":"2008-07-30T14:54:34","guid":{"rendered":"http:\/\/t.motd.kr\/articles\/2219\/filling-the-gap-between-blocking-io-and-nio"},"modified":"2022-12-28T01:45:42","modified_gmt":"2022-12-27T16:45:42","slug":"filling-the-gap-between-blocking-i-o-and-nio","status":"publish","type":"post","link":"https:\/\/vault.motd.kr\/wordpress\/posts\/2219\/filling-the-gap-between-blocking-i-o-and-nio\/","title":{"rendered":"Filling the gap between blocking I\/O and NIO"},"content":{"rendered":"\n<p>A non-blocking <span class=\"caps\">NIO<\/span> <code>Channel<\/code> and a blocking <code>InputStream<\/code> have inevitable impedance mismatch. Because an <code>InputStream<\/code> is supposed to block for every read operation unless there\u2019s some data available in the buffer, any <code>InputStream<\/code>-based decoder implementation can\u2019t be used with a non-blocking <span class=\"caps\">NIO<\/span> application right away.<\/p>\n\n\n\n<p>A common workaround is to prepend a length field for each message so you can wait until you read a whole message before calling <code>InputStream.read()<\/code>. However, this turns your <span class=\"caps\">NIO<\/span> application incompatible with a legacy blocking I\/O application because the legacy application doesn\u2019t prepend a length field at all. You might have managed to modify the legacy application to prepend a length field, but we know it\u2019s not always the case. We need something to fill this gap between two I\/O paradigms.<\/p>\n\n\n\n<p>An <code>ObjectInput\/OutputStream<\/code>-based blocking I\/O network applications are the most common case because it was considered to be the easiest quick-and-dirty solution for intranet Java object exchange. It\u2019s as simple as wrapping an <code>InputStream<\/code> of a <code>Socket<\/code> with an <code>ObjectInputStream<\/code> (i.e. <code>in = new ObjectInputStream(socket.getInputStream());<\/code>).<\/p>\n\n\n\n<p>How can we implement a <span class=\"caps\">NIO<\/span> network application to be interoperable with those legacy applications without any modification? It was considered to be impossible\u2026 until today!<\/p>\n\n\n\n<p>I\u2019ve just released a new milestone of <a href=\"http:\/\/netty.googlecode.com\/\">Netty<\/a> which addresses the issue I described above. It provides <code>CompatibleObjectEncoder<\/code> and <code>CompatibleObjectDecoder<\/code>, which retains interoperability with the legacy <code>ObjectInput\/OutputStream<\/code>-based socket applications.<\/p>\n\n\n\n<p>You will also find you can do the same for any kind of <code>InputStream<\/code> implementations with Netty\u2019s <code>ReplayingDecoder<\/code> with fairly small amount of effort, which means you can shift the paradigm of your complicated blocking protocol client\/server to more scalable non-blocking paradigm while retaining most legacy code.<\/p>\n\n\n\n<p>The excitement of <code>ReplayingDecoder<\/code> doesn\u2019t stop here. It also allows you to implement a non-blocking decoder in a blocking paradigm. In a non-blocking paradigm, you always had to check if there\u2019s enough data in the buffer, like the following:<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>public boolean decode(ByteBuffer in) {\n  if (in.remaining() &lt; 4) {\n    return false;\n  }\n\n  \/\/ Read the length header.\n  int position = in.position();\n  int length = in.getInt();\n  if (in.remaining() &lt; length) {\n    in.position(position);\n    return false;\n  }\n\n  \/\/ Read the body.\n  byte&#91;] data = new byte&#91;length];\n  in.get(data);\n  ...\n  return true;\n}<\/code><\/pre>\n\n\n\n<p>With <code>ReplayingDecoder<\/code>, you don\u2019t need to check the availability of the input buffer at all:<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>public void decode(ByteBuffer in) {\n  \/\/ Read the length header.\n  int length = in.getInt();\n\n  \/\/ Read the body.\n  byte&#91;] data = new byte&#91;length];\n  in.get(data);\n  ...\n}<\/code><\/pre>\n\n\n\n<p>How could this work? <code>ReplayingDecoder<\/code> uses a sort of continuation technique. It rewinds the buffer position to the beginning when there\u2019s not enough data in the buffer automatically and calls <code>decode()<\/code> again (i.e. replays the decode) when more data is received from a remote peer.<\/p>\n\n\n\n<p>You might think this is pretty inefficient, but it turned out to be <a href=\"http:\/\/code.google.com\/p\/netty\/issues\/detail?id=62#c8\">very efficient<\/a> in most cases. Higher throughput means lower chance of replay because we will receive more than one message for a single input buffer (often dozens). Consequently, most messages will be decoded in one shot without a replay. In case of slow connection, it will be less than optimal but you won\u2019t see much difference because it\u2019s already slow because the connection itself is slow. Just compare the code complexity of the two paradigms. I\u2019d definitely go for the latter.<\/p>\n","protected":false},"excerpt":{"rendered":"<p>A non-blocking NIO Channel and a blocking InputStream have inevitable impedance mismatch. Because an InputStream is supposed to block for every read operation unless there\u2019s some data available in the buffer, any InputStream-based decoder implementation can\u2019t be used with a non-blocking NIO application right away. A common workaround is to prepend a length field for&#8230; <a class=\"more-link\" href=\"https:\/\/vault.motd.kr\/wordpress\/posts\/2219\/filling-the-gap-between-blocking-i-o-and-nio\/\">Continue reading <span class=\"meta-nav\">&#8594;<\/span><\/a><\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[1],"tags":[],"yoast_head":"<title>Filling the gap between blocking I\/O and NIO &#8212; T&#039;s message of the day<\/title>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/vault.motd.kr\/wordpress\/posts\/2219\/filling-the-gap-between-blocking-i-o-and-nio\/\" \/>\n<meta property=\"og:locale\" content=\"en_US\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"Filling the gap between blocking I\/O and NIO &#8212; T&#039;s message of the day\" \/>\n<meta property=\"og:description\" content=\"A non-blocking NIO Channel and a blocking InputStream have inevitable impedance mismatch. Because an InputStream is supposed to block for every read operation unless there\u2019s some data available in the buffer, any InputStream-based decoder implementation can\u2019t be used with a non-blocking NIO application right away. A common workaround is to prepend a length field for... Continue reading &#8594;\" \/>\n<meta property=\"og:url\" content=\"https:\/\/vault.motd.kr\/wordpress\/posts\/2219\/filling-the-gap-between-blocking-i-o-and-nio\/\" \/>\n<meta property=\"og:site_name\" content=\"T&#039;s message of the day\" \/>\n<meta property=\"article:publisher\" content=\"https:\/\/www.facebook.com\/trustin\" \/>\n<meta property=\"article:author\" content=\"https:\/\/www.facebook.com\/trustin\" \/>\n<meta property=\"article:published_time\" content=\"2008-07-30T14:54:34+00:00\" \/>\n<meta property=\"article:modified_time\" content=\"2022-12-27T16:45:42+00:00\" \/>\n<meta name=\"author\" content=\"Trustin Lee\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:creator\" content=\"@https:\/\/www.twitter.com\/trustin\" \/>\n<meta name=\"twitter:site\" content=\"@trustin\" \/>\n<meta name=\"twitter:label1\" content=\"Written by\" \/>\n\t<meta name=\"twitter:data1\" content=\"Trustin Lee\" \/>\n\t<meta name=\"twitter:label2\" content=\"Est. reading time\" \/>\n\t<meta name=\"twitter:data2\" content=\"3 minutes\" \/>\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\"@context\":\"https:\/\/schema.org\",\"@graph\":[{\"@type\":\"Article\",\"@id\":\"https:\/\/vault.motd.kr\/wordpress\/posts\/2219\/filling-the-gap-between-blocking-i-o-and-nio\/#article\",\"isPartOf\":{\"@id\":\"https:\/\/vault.motd.kr\/wordpress\/posts\/2219\/filling-the-gap-between-blocking-i-o-and-nio\/\"},\"author\":{\"name\":\"Trustin Lee\",\"@id\":\"https:\/\/vault.motd.kr\/wordpress\/#\/schema\/person\/4430ad90fc2ddeef051565701d85db9d\"},\"headline\":\"Filling the gap between blocking I\/O and NIO\",\"datePublished\":\"2008-07-30T14:54:34+00:00\",\"dateModified\":\"2022-12-27T16:45:42+00:00\",\"mainEntityOfPage\":{\"@id\":\"https:\/\/vault.motd.kr\/wordpress\/posts\/2219\/filling-the-gap-between-blocking-i-o-and-nio\/\"},\"wordCount\":476,\"publisher\":{\"@id\":\"https:\/\/vault.motd.kr\/wordpress\/#\/schema\/person\/4430ad90fc2ddeef051565701d85db9d\"},\"inLanguage\":\"en-US\"},{\"@type\":\"WebPage\",\"@id\":\"https:\/\/vault.motd.kr\/wordpress\/posts\/2219\/filling-the-gap-between-blocking-i-o-and-nio\/\",\"url\":\"https:\/\/vault.motd.kr\/wordpress\/posts\/2219\/filling-the-gap-between-blocking-i-o-and-nio\/\",\"name\":\"Filling the gap between blocking I\/O and NIO &#8212; T&#039;s message of the day\",\"isPartOf\":{\"@id\":\"https:\/\/vault.motd.kr\/wordpress\/#website\"},\"datePublished\":\"2008-07-30T14:54:34+00:00\",\"dateModified\":\"2022-12-27T16:45:42+00:00\",\"breadcrumb\":{\"@id\":\"https:\/\/vault.motd.kr\/wordpress\/posts\/2219\/filling-the-gap-between-blocking-i-o-and-nio\/#breadcrumb\"},\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"ReadAction\",\"target\":[\"https:\/\/vault.motd.kr\/wordpress\/posts\/2219\/filling-the-gap-between-blocking-i-o-and-nio\/\"]}]},{\"@type\":\"BreadcrumbList\",\"@id\":\"https:\/\/vault.motd.kr\/wordpress\/posts\/2219\/filling-the-gap-between-blocking-i-o-and-nio\/#breadcrumb\",\"itemListElement\":[{\"@type\":\"ListItem\",\"position\":1,\"name\":\"Home\",\"item\":\"https:\/\/vault.motd.kr\/wordpress\/\"},{\"@type\":\"ListItem\",\"position\":2,\"name\":\"Filling the gap between blocking I\/O and NIO\"}]},{\"@type\":\"WebSite\",\"@id\":\"https:\/\/vault.motd.kr\/wordpress\/#website\",\"url\":\"https:\/\/vault.motd.kr\/wordpress\/\",\"name\":\"T's message of the day\",\"description\":\"the best is yet to come\",\"publisher\":{\"@id\":\"https:\/\/vault.motd.kr\/wordpress\/#\/schema\/person\/4430ad90fc2ddeef051565701d85db9d\"},\"potentialAction\":[{\"@type\":\"SearchAction\",\"target\":{\"@type\":\"EntryPoint\",\"urlTemplate\":\"https:\/\/vault.motd.kr\/wordpress\/?s={search_term_string}\"},\"query-input\":\"required name=search_term_string\"}],\"inLanguage\":\"en-US\"},{\"@type\":[\"Person\",\"Organization\"],\"@id\":\"https:\/\/vault.motd.kr\/wordpress\/#\/schema\/person\/4430ad90fc2ddeef051565701d85db9d\",\"name\":\"Trustin Lee\",\"image\":{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\/\/vault.motd.kr\/wordpress\/#\/schema\/person\/image\/\",\"url\":\"https:\/\/vault.motd.kr\/wordpress\/wp-content\/uploads\/2020\/09\/avatar-2019-966px.png\",\"contentUrl\":\"https:\/\/vault.motd.kr\/wordpress\/wp-content\/uploads\/2020\/09\/avatar-2019-966px.png\",\"width\":966,\"height\":966,\"caption\":\"Trustin Lee\"},\"logo\":{\"@id\":\"https:\/\/vault.motd.kr\/wordpress\/#\/schema\/person\/image\/\"},\"sameAs\":[\"https:\/\/t.motd.kr\/\",\"https:\/\/www.facebook.com\/trustin\",\"https:\/\/www.instagram.com\/trustinlee\/\",\"https:\/\/www.linkedin.com\/in\/trustin\",\"https:\/\/twitter.com\/https:\/\/www.twitter.com\/trustin\",\"https:\/\/www.youtube.com\/@trustinlee\"]}]}<\/script>","yoast_head_json":{"title":"Filling the gap between blocking I\/O and NIO &#8212; T&#039;s message of the day","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/vault.motd.kr\/wordpress\/posts\/2219\/filling-the-gap-between-blocking-i-o-and-nio\/","og_locale":"en_US","og_type":"article","og_title":"Filling the gap between blocking I\/O and NIO &#8212; T&#039;s message of the day","og_description":"A non-blocking NIO Channel and a blocking InputStream have inevitable impedance mismatch. Because an InputStream is supposed to block for every read operation unless there\u2019s some data available in the buffer, any InputStream-based decoder implementation can\u2019t be used with a non-blocking NIO application right away. A common workaround is to prepend a length field for... Continue reading &#8594;","og_url":"https:\/\/vault.motd.kr\/wordpress\/posts\/2219\/filling-the-gap-between-blocking-i-o-and-nio\/","og_site_name":"T&#039;s message of the day","article_publisher":"https:\/\/www.facebook.com\/trustin","article_author":"https:\/\/www.facebook.com\/trustin","article_published_time":"2008-07-30T14:54:34+00:00","article_modified_time":"2022-12-27T16:45:42+00:00","author":"Trustin Lee","twitter_card":"summary_large_image","twitter_creator":"@https:\/\/www.twitter.com\/trustin","twitter_site":"@trustin","twitter_misc":{"Written by":"Trustin Lee","Est. reading time":"3 minutes"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"Article","@id":"https:\/\/vault.motd.kr\/wordpress\/posts\/2219\/filling-the-gap-between-blocking-i-o-and-nio\/#article","isPartOf":{"@id":"https:\/\/vault.motd.kr\/wordpress\/posts\/2219\/filling-the-gap-between-blocking-i-o-and-nio\/"},"author":{"name":"Trustin Lee","@id":"https:\/\/vault.motd.kr\/wordpress\/#\/schema\/person\/4430ad90fc2ddeef051565701d85db9d"},"headline":"Filling the gap between blocking I\/O and NIO","datePublished":"2008-07-30T14:54:34+00:00","dateModified":"2022-12-27T16:45:42+00:00","mainEntityOfPage":{"@id":"https:\/\/vault.motd.kr\/wordpress\/posts\/2219\/filling-the-gap-between-blocking-i-o-and-nio\/"},"wordCount":476,"publisher":{"@id":"https:\/\/vault.motd.kr\/wordpress\/#\/schema\/person\/4430ad90fc2ddeef051565701d85db9d"},"inLanguage":"en-US"},{"@type":"WebPage","@id":"https:\/\/vault.motd.kr\/wordpress\/posts\/2219\/filling-the-gap-between-blocking-i-o-and-nio\/","url":"https:\/\/vault.motd.kr\/wordpress\/posts\/2219\/filling-the-gap-between-blocking-i-o-and-nio\/","name":"Filling the gap between blocking I\/O and NIO &#8212; T&#039;s message of the day","isPartOf":{"@id":"https:\/\/vault.motd.kr\/wordpress\/#website"},"datePublished":"2008-07-30T14:54:34+00:00","dateModified":"2022-12-27T16:45:42+00:00","breadcrumb":{"@id":"https:\/\/vault.motd.kr\/wordpress\/posts\/2219\/filling-the-gap-between-blocking-i-o-and-nio\/#breadcrumb"},"inLanguage":"en-US","potentialAction":[{"@type":"ReadAction","target":["https:\/\/vault.motd.kr\/wordpress\/posts\/2219\/filling-the-gap-between-blocking-i-o-and-nio\/"]}]},{"@type":"BreadcrumbList","@id":"https:\/\/vault.motd.kr\/wordpress\/posts\/2219\/filling-the-gap-between-blocking-i-o-and-nio\/#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https:\/\/vault.motd.kr\/wordpress\/"},{"@type":"ListItem","position":2,"name":"Filling the gap between blocking I\/O and NIO"}]},{"@type":"WebSite","@id":"https:\/\/vault.motd.kr\/wordpress\/#website","url":"https:\/\/vault.motd.kr\/wordpress\/","name":"T's message of the day","description":"the best is yet to come","publisher":{"@id":"https:\/\/vault.motd.kr\/wordpress\/#\/schema\/person\/4430ad90fc2ddeef051565701d85db9d"},"potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/vault.motd.kr\/wordpress\/?s={search_term_string}"},"query-input":"required name=search_term_string"}],"inLanguage":"en-US"},{"@type":["Person","Organization"],"@id":"https:\/\/vault.motd.kr\/wordpress\/#\/schema\/person\/4430ad90fc2ddeef051565701d85db9d","name":"Trustin Lee","image":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/vault.motd.kr\/wordpress\/#\/schema\/person\/image\/","url":"https:\/\/vault.motd.kr\/wordpress\/wp-content\/uploads\/2020\/09\/avatar-2019-966px.png","contentUrl":"https:\/\/vault.motd.kr\/wordpress\/wp-content\/uploads\/2020\/09\/avatar-2019-966px.png","width":966,"height":966,"caption":"Trustin Lee"},"logo":{"@id":"https:\/\/vault.motd.kr\/wordpress\/#\/schema\/person\/image\/"},"sameAs":["https:\/\/t.motd.kr\/","https:\/\/www.facebook.com\/trustin","https:\/\/www.instagram.com\/trustinlee\/","https:\/\/www.linkedin.com\/in\/trustin","https:\/\/twitter.com\/https:\/\/www.twitter.com\/trustin","https:\/\/www.youtube.com\/@trustinlee"]}]}},"_links":{"self":[{"href":"https:\/\/vault.motd.kr\/wordpress\/wp-json\/wp\/v2\/posts\/2219"}],"collection":[{"href":"https:\/\/vault.motd.kr\/wordpress\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/vault.motd.kr\/wordpress\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/vault.motd.kr\/wordpress\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/vault.motd.kr\/wordpress\/wp-json\/wp\/v2\/comments?post=2219"}],"version-history":[{"count":3,"href":"https:\/\/vault.motd.kr\/wordpress\/wp-json\/wp\/v2\/posts\/2219\/revisions"}],"predecessor-version":[{"id":5959,"href":"https:\/\/vault.motd.kr\/wordpress\/wp-json\/wp\/v2\/posts\/2219\/revisions\/5959"}],"wp:attachment":[{"href":"https:\/\/vault.motd.kr\/wordpress\/wp-json\/wp\/v2\/media?parent=2219"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/vault.motd.kr\/wordpress\/wp-json\/wp\/v2\/categories?post=2219"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/vault.motd.kr\/wordpress\/wp-json\/wp\/v2\/tags?post=2219"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}