Install Hadoop 2.2.0 on Ubuntu 12.0.4 LTS

I am starting to learn and study harder on the Big Data and Analytics area. You may notice from now on a shift from this blog from a more Microsoft-centric approach to a more mixed use, Open Source snippets followed by hints on Microsoft services I work with for years.

Let's start with Hadoop!

I will here only post a great tutorial: http://codesfusion.blogspot.com.br/2013/10/setup-hadoop-2x-220-on-ubuntu.html?m=1

This tutorial was applied with a 100% accuracy on Ubuntu 12.0.4. Enjoy!

Change DFS-R default debug logs size and paths

The debug logs are usually left alone and unused... until you have a strange issue in your replication. If you have a contract with Microsoft, bet they will ask you for these logs, and they contain really valuable information, if you decrypt the not easy to understand format and messages.

According to your data size and amount of changes, the defaults are not suitable. If you have TBs of replicated data being modified quite intensely the default values will not be enough, and the 1000 logs retained will last for hours or max one or two days.

So, here the tricks:

1.  How to increase the number of DFS-R debug logs?

The answer is in the following kbarticle:

http://support.microsoft.com/kb/958893

SETTING: Debug Log Messages

Default: 200000

Range: 1000 to 4294967295 (FFFFFFFF)

WMIC syntax: wmic /namespace:\\root\microsoftdfs path dfsrmachineconfig set maxdebuglogmessages=1000000

How to change the path were they are saved?

SETTING: Debug Log File Path

Default: %windir%\debug

WMIC syntax: wmic /namespace:\\root\microsoftdfs path dfsrmachineconfig set debuglogfilepath="d:\dfsrlogs"

So, with this blog we start 2014!