http://wiki.iiis.systems:9988/w/index.php?action=history&feed=atom&title=User%3AShenh10 2026-04-05T11:33:55Z Revision history for this page on the wiki MediaWiki 1.24.0-rc.1 http://wiki.iiis.systems:9988/w/index.php?title=User:Shenh10&diff=419&oldid=prev 2016-11-17T03:46:54Z

<p>Blanked the page</p> <table class='diff diff-contentalign-left'> <col class='diff-marker' /> <col class='diff-content' /> <col class='diff-marker' /> <col class='diff-content' /> <tr style='vertical-align: top;'> <td colspan='2' style="background-color: white; color:black; text-align: center;">← Older revision</td> <td colspan='2' style="background-color: white; color:black; text-align: center;">Revision as of 03:46, 17 November 2016</td> </tr><tr><td colspan="2" class="diff-lineno">Line 1:</td> <td colspan="2" class="diff-lineno">Line 1:</td></tr> <tr><td class='diff-marker'>−</td><td style="color:black; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #ffe49c; vertical-align: top; white-space: pre-wrap;"><div><del style="font-weight: bold; text-decoration: none;">When choosing cloud platform such as OpenStack to deploy a computing framework, the main challenge exists in efficiency. To store processing data inside Virtual Machines is neither reliable nor&#160; collaborative.</del></div></td><td colspan="2">&#160;</td></tr> <tr><td class='diff-marker'>&#160;</td><td style="background-color: #f9f9f9; color: #333333; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #e6e6e6; vertical-align: top; white-space: pre-wrap;"></td><td class='diff-marker'>&#160;</td><td style="background-color: #f9f9f9; color: #333333; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #e6e6e6; vertical-align: top; white-space: pre-wrap;"></td></tr> <tr><td class='diff-marker'>−</td><td style="color:black; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #ffe49c; vertical-align: top; white-space: pre-wrap;"><div><del style="font-weight: bold; text-decoration: none;">However, deploying a shared filesystem on physical machine leads to bottleneck of network bandwidth at the Neutron node. We promoted a solution achieving better networking performance in such a deploy case.&#160; By hacking into OVS and OVS Agent, I have achieved Hadoop's layer2 access to physical HDFS which improved the locality and avoided layer3 router congestion. Furthermore, Docker can also utilize this solution to get switching performance comparing to Linux Iptables forwarding. Based on such optimized platforms of 100+ nodes cluster, I have done a benchmark on how Hadoop behaves when deployed on virtualization platforms(</del></div></td><td colspan="2">&#160;</td></tr> <tr><td class='diff-marker'>−</td><td style="color:black; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #ffe49c; vertical-align: top; white-space: pre-wrap;"><div><del style="font-weight: bold; text-decoration: none;">Baremental, OpenStack, Docker)</del></div></td><td colspan="2">&#160;</td></tr> <tr><td class='diff-marker'>−</td><td style="color:black; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #ffe49c; vertical-align: top; white-space: pre-wrap;"><div><del style="font-weight: bold; text-decoration: none;"></del></div></td><td colspan="2">&#160;</td></tr> <tr><td class='diff-marker'>−</td><td style="color:black; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #ffe49c; vertical-align: top; white-space: pre-wrap;"><div><del style="font-weight: bold; text-decoration: none;"></del></div></td><td colspan="2">&#160;</td></tr> <tr><td class='diff-marker'>−</td><td style="color:black; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #ffe49c; vertical-align: top; white-space: pre-wrap;"><div><del style="font-weight: bold; text-decoration: none;">A set of bash scripts goes here https://github.com/iiisthu/HadoopBenchmark</del></div></td><td colspan="2">&#160;</td></tr> <tr><td class='diff-marker'>−</td><td style="color:black; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #ffe49c; vertical-align: top; white-space: pre-wrap;"><div><del style="font-weight: bold; text-decoration: none;"></del></div></td><td colspan="2">&#160;</td></tr> <tr><td class='diff-marker'>−</td><td style="color:black; font-size: 88%; border-style: solid; border-width: 1px 1px 1px 4px; border-radius: 0.33em; border-color: #ffe49c; vertical-align: top; white-space: pre-wrap;"><div><del style="font-weight: bold; text-decoration: none;">And a template for deploy Hadoop in Openstack Virtual Machines goes here https://github.com/iiisthu/HadoopTemplate</del></div></td><td colspan="2">&#160;</td></tr> </table>

Shenh10 http://wiki.iiis.systems:9988/w/index.php?title=User:Shenh10&diff=408&oldid=prev 2016-11-17T02:46:32Z

<p>Created page with &quot;When choosing cloud platform such as OpenStack to deploy a computing framework, the main challenge exists in efficiency. To store processing data inside Virtual Machines is ne...&quot;</p> <p><b>New page</b></p><div>When choosing cloud platform such as OpenStack to deploy a computing framework, the main challenge exists in efficiency. To store processing data inside Virtual Machines is neither reliable nor collaborative.<br /> <br /> However, deploying a shared filesystem on physical machine leads to bottleneck of network bandwidth at the Neutron node. We promoted a solution achieving better networking performance in such a deploy case. By hacking into OVS and OVS Agent, I have achieved Hadoop's layer2 access to physical HDFS which improved the locality and avoided layer3 router congestion. Furthermore, Docker can also utilize this solution to get switching performance comparing to Linux Iptables forwarding. Based on such optimized platforms of 100+ nodes cluster, I have done a benchmark on how Hadoop behaves when deployed on virtualization platforms(<br /> Baremental, OpenStack, Docker)<br /> <br /> <br /> A set of bash scripts goes here https://github.com/iiisthu/HadoopBenchmark<br /> <br /> And a template for deploy Hadoop in Openstack Virtual Machines goes here https://github.com/iiisthu/HadoopTemplate</div>

Shenh10