Install Hadoop 2.2.0 on Ubuntu 12.0.4 LTS

I am starting to learn and study harder on the Big Data and Analytics area. You may notice from now on a shift from this blog from a more Microsoft-centric approach to a more mixed use, Open Source snippets followed by hints on Microsoft services I work with for years.

Let's start with Hadoop!

I will here only post a great tutorial: http://codesfusion.blogspot.com.br/2013/10/setup-hadoop-2x-220-on-ubuntu.html?m=1

This tutorial was applied with a 100% accuracy on Ubuntu 12.0.4. Enjoy!

Change DFS-R default debug logs size and paths

The debug logs are usually left alone and unused... until you have a strange issue in your replication. If you have a contract with Microsoft, bet they will ask you for these logs, and they contain really valuable information, if you decrypt the not easy to understand format and messages.

According to your data size and amount of changes, the defaults are not suitable. If you have TBs of replicated data being modified quite intensely the default values will not be enough, and the 1000 logs retained will last for hours or max one or two days.

So, here the tricks:

1.  How to increase the number of DFS-R debug logs?

The answer is in the following kbarticle:

http://support.microsoft.com/kb/958893

SETTING: Debug Log Messages

Default: 200000

Range: 1000 to 4294967295 (FFFFFFFF)

WMIC syntax: wmic /namespace:\\root\microsoftdfs path dfsrmachineconfig set maxdebuglogmessages=1000000

How to change the path were they are saved?

SETTING: Debug Log File Path

Default: %windir%\debug

WMIC syntax: wmic /namespace:\\root\microsoftdfs path dfsrmachineconfig set debuglogfilepath="d:\dfsrlogs"

So, with this blog we start 2014! 

Recover a SSL Certificate lost private key on Windows Server 2008 R2

So you get the certificate, import it to the Computer Certificates store, and then you want to use it on IIS or other application which requires a certificate, like, Forefront TMG.

And you notice that your recently added certificate is not there. To recover it, just follow the steps provided by Entrust on this very easy to follow how to: http://www.entrust.net/knowledge-base/technote.cfm?tn=7905

Binding a SSL Certificate to Website Error: HRESULT: 0x80070520

Hello,

today I faced a problem with IIS and Certificates. The error was:

HRESULT: 0x80070520

I found a very good article here: http://blogs.msdn.com/b/kaushal/archive/2012/10/07/error-hresult-0x80070520-when-adding-ssl-binding-in-iis.aspx

In my specific case the problem was the permissions. I had to remove the certificate and set Administrators as Full Control to 'THIS FOLDER ONLY' and Everyone with Full Control over the entire folder.

After that imported the certificate again and it worked.

I am a bit worried with the Everyone permission, but this was the recommendation from this article: http://msdn.microsoft.com/en-us/library/ee248638%28v=vs.100%29.aspx

And, another problem solved!

Export Installed Updates in Windows Server 2008 or 2008 R2

There is a quick command to export in a nice CSV format all installed updates in a Windows Server 2008 or 2008 R2 system:

1. Open a command prompt

2. Type: wmic qfe get /format:csv >C:\updates.csv

What is the benefit of it? This is a way to build a Changelog to track the changes performed in the system. Each Month or Scheduled update 'day' export the changes, so you can see the history of applied patches.

Find the Shares from your DFS Namespace

DFS Namespace information can be easily exported to a nice XML file with the command:

dfsutil root export \\domain\namespacename C:\Export.xml

But now you have a XML file, which is not the best file type to work with and check important info. Basically what I will show you is a small script in Powershell that parses the XML and return the ONLINE targets sharename, but you can adapt to return the info you need.

# Parsing XML from Namespace
[xml]$dfs = Get-Content output.xml
$output = @()
foreach ($link in $dfs.Root.Link)
{
    if ($link.Name -match "DEEI\\Groups")
    {
        $user = $link.Name.Split("\")


        foreach ($target in $link.Target)
        {
            $shareName = $target.get_InnerXml()
            if ($target.state -eq "ONLINE")
            {       
                $output+=$sharename
            }
        }
    }
}       
$output | Out-File '.\OUTPUT-SHARE.TXT'

The small script above returns all shares which are ONLINE and output them to the OUTPUT-SHARE.TXT. It takes as input the file 'output.xml' which is the xml from the namespace.

Feel free to adapt it. Please let your comment.

DFS-R Replication problems: Event ID 6002

Hi,

today morning, while applying windows updates into one of mine DFS-R and DFS-N servers, I found a strange ID, 6002:


The DFS Replication service detected invalid msDFSR-Subscriber object data while polling for configuration information.

Additional Information:
Object DN: CN=<removed>,CN=DFSR-LocalSettings,CN=<removed>,OU=<removed>,OU=<removed>,OU=<removed>,OU=<removed>,DC=<domain name>,DC=com
Attribute Name: msDFSR-MemberReference
Domain Controller: <DC hostname>
Polling Cycle: 60 minutes

After seeing this error I filtered mine alerts, and found that for 3 days these alerts were being shown, just a few each day. So, let´s do what every SysAdmin does: google it!

I had the feeling that this was related to conflicts or orphan entries in LDAP DB that contains the entire DFS configuration, and the feeling became true.

My Domain is  based on Windows Server 2008 R2 with 2008 R2 Functional Level.

I found some interesting entries:

http://www.chicagotech.net/netforums/viewtopic.php?p=4675&sid=4533d81e7d24afa689686ff5d5ffbdb3
http://www.eventid.net/display-eventid-6002-source-DFSR-eventno-10483-phase-1.htm
http://social.technet.microsoft.com/wiki/contents/articles/1158.dfsr-event-6002-dfs-replication.aspx (fix only for 2k3 R2 - other problem)

and here the details about the ms-DFSR-MemberReference attribute (Windows):
http://msdn.microsoft.com/en-us/library/windows/desktop/ms677157(v=vs.85).aspx#win_2008_r2

But, after the links and the intro, what was the solution?

Before going to the solution, I need to add one more thing: this error stopped replication for ALL folders and groups. So this is a very critical problem. I created some Diag Reports and create some test files to see them replicated, but nothing happened for hours.

Steps taken:

1. Get the errors (look in all involved servers from your DFS-R farm, if you replicate folders from A to B, look on A and B for 6002 errors - filter the event viewer)

2. Go to your Domain Controller (I am assuming you have DFS deployed in a Domain), open the ADSIEdit.msc console (this tool allows you to browse the LDAP structure from your domain, including all attributes from the objects, allowing a RW view... be careful when editing this, this is only required for advanced procedures)

3. In the ADSIEdit console, connect to your DC/Domain and navigate to the key informed in the error. Open the details for the key, there you will find the replicated folder name.

4. Do the same in the other servers. Verify in all the servers the replicated folder name

5. Do not remove the entry yet. Go back to your DFS Management snap-in and delete the affected replicated group.

6. Wait for the changes to propagate, to speedup run in an elevated prompt dfsrdiag PollAD /Member:<your  DC hostname>

7. Wait up to 1h to see the changes propagated. For me, after removing the conflicting Replication Group from the DFS-R Management (not removing from view, but deleting the RG totally) the DFS replication started to work properly again (yes!!!). But the conflicted still existed in MSIEdit.

8. To stop the 6002 errors, I deleted the entries from LDAP DB and restarted DFS-R service. After that everything was OK.

So, one more problem solved and documented so you can also benefit from this. Please comment