The definitive guide of connecting Apache via LDAP SSL to ActiveDirectory + Subversion

Being taken though hell I of configuring the damn authentication mechanism.
I needed to integrate Apache with ActiveDirectory via LDAP SSL. The manual SUCKS! It does not say anything useful, except for the syntax of the directives.
This article was the most definitive of configuring the integration. But lacks the information about how to make the SSL work correctly.

Here's the kicker, it is as simple as the following several elements(this is in the top of /etc/apache2/sites-available/default):
# Make sure ve don't care about the server's certificate, because we don't
LDAPVerifyServerCert off
LDAPTrustedMode SSL
# The server's client cert information: the cert and the matching private key
LDAPTrustedGlobalCert CERT_BASE64 /etc/apache2/sites-available/cert1.pem
LDAPTrustedGlobalCert KEY_BASE64 /etc/apache2/sites-available/key1.pem

These go into the ROOT. Do not try to put them in the Location, nor Directory, nor VirtualHost.

And the main change to enable the SSL transport:
AuthLDAPURL "ldaps://adserver.example:636/DC=adserver,DC=example?sAMAccountName?sub?(objectClass=*)" SSL

These are the steps:

1. Create the client Key and Certificate

The cert1.pem and key1.pem are created like described here:

openssl genrsa 1024 > key1.pem
openssl req -new -x509 -nodes -sha1 -days 365 -key key1.pem > cert1.pem

For an additional configuration reduction bonus you can have it in one single file:
cat cert1.pem key1.pem > pcert.pem
LDAPTrustedGlobalCert CERT_BASE64 /etc/apache2/sites-available/pcert.pem

2. Enable the correct modules on Apache HTTP Server 2.2.

On my Ubuntu system the module enabling is done like this:

sudo a2enmod alias

sudo a2enmod auth_basic

sudo a2enmod authnz_ldap

sudo a2enmod authz_default

sudo a2enmod authz_user

sudo a2enmod dav

sudo a2enmod dav_svn

sudo a2enmod ldap

Or uncomment these elements in the httpd.conf:
LoadModule actions_module modules/mod_actions.so
LoadModule auth_basic_module modules/mod_auth_basic.so
LoadModule authn_default_module modules/mod_authn_default.so
LoadModule authnz_ldap_module modules/mod_authnz_ldap.so
LoadModule authz_host_module modules/mod_authz_host.so
LoadModule authz_user_module modules/mod_authz_user.so
LoadModule ldap_module modules/mod_ldap.so

And the modules that are prescribed by the Subversion.


3. The access restrictions and Subversion

The short info for configuring the access restrictions.
This will allow people in the users in SVN Writers to commit and SVN Readers will be able to checkout and connect.

<Location>
# This enables Subversion
DAV svn
# Location of the Subversion repository
SVNPath /home/ldaptest/

# How we are going to get authenticated
AuthBasicProvider ldap
AuthType Basic
AuthzLDAPAuthoritative on
AuthName "My Subversion server"
#The URL of the ActiveDirectory server
AuthLDAPURL "ldaps://adserver.example:636/DC=adserver,DC=example?sAMAccountName?sub?(objectClass=*)" SSL
# Credentials for the Apache HTTP to connect to the A/D to issue queries
AuthLDAPBindDN "subversion@adserver.example"
AuthLDAPBindPassword 555

# Limit all write operations to users within SVN Writers group
<LimitExcept>
require ldap-group CN=SVN Writers,CN=Users,DC=adserver,DC=example
</LimitExcept>
# Limit logon and reading to only users in SVN Readers group
require ldap-group CN=SVN Readers,CN=Users,DC=adserver,DC=example
</Location>

4. Add the following to the default site on Apache HTTP

The file is /etc/apache2/sites-available/default
Right at the TOP.

# Make sure ve don't care about the server's certificate, because we don't
LDAPVerifyServerCert off
LDAPTrustedMode SSL
# The server's client cert information: the cert and the matching private key
LDAPTrustedGlobalCert CERT_BASE64 /etc/apache2/sites-available/cert1.pem
LDAPTrustedGlobalCert KEY_BASE64 /etc/apache2/sites-available/key1.pem

Be careful how you truncate...

Here's one example how truncation has made the entry look like it's a conspiracy theorists' website:
The text below reads:"...See the video at the end for the Test Results! NOT...posted by Kipay".

This one is from Instructibles

5 signs of a hijacked client

The context rant

While doing my main line of work, I never shy away from doing some consultancy work for clients. And I mean no programming, I mean business operation consultancy. Though I am a application development consultant and application developer, I do have enough experience to identify when a company has issues with overhead in work. I mean premature optimization is what a lot of programmers are all about and sometimes the business side benefits from that very well.

For the topic at hand. I had this good friend nagging me for a long amount of time for a consult. He is a successful businessman and no one in their right mind would call his business a failure. It's a nice Anonymous LTD that manages to turn over a few million euros per year with 12 people. So I signed up. I do not consider charging my friends for any non technical work. Since I follow the ideology of having a lot of friends at the expense of quick income.

The "motherload"

So I started rather quickly on analyzing Anonymous LTD. The analysis part was quite fun and easy. I liked talking to people and they gladly shared all of the positives and negatives of their work. Some even had suggestions :) And that is where I started to suspect that something was wrong.

1. The employees are de facto subordinate non management staff.
   The sales people were "afraid"(read subordinate) of not the management, but to the accounting. Yeah, that is when I had the first WTF moment.
2. There is a department that everyone tries to please, and not the management.
   The sales and other employees felt that they were somehow more dependent on accounting rather then the strategical management. Remember, accounting is a support operation in non accounting related companies.
3. The management displays uncertainty and consults some department on business matters.(Legal and purely technical are different stories)
   Really, when I went on to ask why the hell was there no CEO in the company, my friend, the director called up head of accounting to ask!!!!
4. You present a solution that improves managerial oversight and operations visibility and it's shot down by management.
   I proposed a new reporting tool/function, that was integrated into the stock tracking and order system. The director and top sales manager said: "Wow! Great! We really want it!". And two days later, they cut out the functionality, because it had not enough financial data(the stock tracking and order system, was not suppose to have any financial info).
5. The management's position changes whenever the company gets new, non management level, people on board.
   I already had the contract secured and signed. The company that was suppose to perform the work was selected and already analyzing requirements. Great! I thought. Then a new person is hired into the sales team. The contract is cancelled, because of hudge requirements changes.

The company had an issue with their stock to quote to order to delivery operation. It is unreliable and overburdened by out of date data. They still don't have any automation of warehouse stock information. Why? Because the new head sales-person wants to see a dashboard with all the client's information orders, payment statuses, credit and in real time and the accounting do not want to let that information out of their hands.

That is what I call a hijacked company:
A company that is taken over by a nonessential department and weak, collaborating, management.
Be very, very careful, it's a trap of uncertainty.

Aftermath

I managed to push trough a CEO for the company, because the managing director was and is my friend and a little too soft. And last time I checked, he still keeps accounting at bay and the director is non managing.

Why is software not that scalable?

(FYI: This is a 3 AM rant, so don't take it too hard, if anyone reads this at all)

Most software out there relies so much on shared data, that it does not easily scale.

The problem with that is, that somehow developers don't understand that most of the time the data may be allowed to be stale(though still relevant) - see how Amazon Dynamo's clients handle it. Or that most data is accessed only by one process at a time.
In fact, when looking at human interaction systems we see that most of the data is stale when people perform a certain task.

Here are some point that I describe below:

• Allow data to be stale, there is very little reason of making sure everything is always up to date all the time, spread the data as much as possible
  
• Check out the data access patterns, most data is access
• Understand that need for data does not involve one process/thread being hung up on one resource
• If you need stuff to be up-to-date in real time, share the responsibility - aggregating data is faster and easier

On the freshness of data.

Sample from most of the governmental institutions: laws and regulations have a time of activation that is shared in advance. A lot of decisions are made at a low level without checking the central(at a national level) legal repository every time. That is when data is stale, but still very much relevant.

Sample from most businesses: business rules are much more dynamic than at the governmental institution, but still we see that most of them change very little. Sure you can change it at once, but you would never expect any of you own businesses employees to act on the changed rules, until the employee gets the message about a change in the rules.

Sample from every society that has laws that change or may be changed: when a law is passed, you don't know that is has changes some other law or made something illegal or legal instantly. It has a date of coming into force. Up until then every person will know most important laws that are in effect and when the new law comes into effect.

Then there is the access to data.

Sample from banking: chances are that there will be no more than ONE "process" using an account data at a time. In fact, I bet that, the absolute majority of operations are performed by one process accessing ONE account. The reality is, that the only part that the bank needs to be absolutely sure about is the "credit" operation(Say bye to "credit->debit|undo transaction"). The debit operation can be reasonably fast in asynchronous mode and can send notification of a failure.

Sample from real life: if you don't find your personal mug in the cupboard you don't just stand at the cupboard until someone returns it! You go and look around. The mug is the data here and the person is the process. Standing and waiting for it ti be returned is equivalent of locking a resource.
As this sample shows, locking and waiting is not always the best option. The reality is, that it's not the process that has taken the resource that locks, it's the waiting resource that locks and waits.

My proposed strategy, is acquire ownership of the data and leave a note stating who knows it's whereabouts.

So the morale is don't lock, but take. Don't wait, but look around.

Now. I can't really comment on systems where real time data is essential. But in medicine real time means means the difference between life and death. But those systems are embarrassingly parallelizable.

Remember, our world works on Eventually Consistent Model, not the, ironically fatally named, ACID one.

Just look around, the world is already massively parallel, why create new ideas when mother nature has provided us with a lot of the answers.

WOW! Moment

Rarely do I go "Wow!". But this is something incredible!
Check out Aurora Concept

Make your Hibernate data objects smart?

In a race to learn Hibernate and create products for my former job, I have found a simple way to make my Hibernate data objects smarter.
We all know th standard way to access the Hibernate session using the SessionHolder*.

This is a basic abstract class(shown below) that will add the .add(), .update() and .delete() methods to your data objects.

Just extend this class by your Hibernate DTO like this:
public class Address extends BaseData

And then just after populating the new Address object, you write address.add().

public abstract class BaseData implements Serializable {
 
    public String add(){
 
        try{
 
            SessionHolder.currentSession().getSess().save(this);
 
            SessionHolder.currentSession().getSess().flush();
 
        }catch(Exception ex){
 
            SessionHolder.endSession();
 
            ex.printStackTrace();
 
            Messenger.addFatalMessage(ex.getMessage(),ex.getMessage());
 
            return StandardResults.FAIL;
 
        }
 
        return StandartResults.SUCCESS;
 
    }
 
    public String update(){
 
        try {
 
            SessionHolder.currentSession().getSess().update(this);
 
            SessionHolder.currentSession().getSess().flush();
 
            return StandartResults.SUCCESS;
 
        } catch (Exception ex) {
 
            SessionHolder.endSession();
 
            ex.printStackTrace();
 
            Messenger.addFatalMessage(ex.getMessage(),ex.getMessage());
 
            return StandardResults.FAIL;
 
        }
 
    }
 
    public String delete(){
 
        try {
 
            SessionHolder.currentSession().getSess().delete(this);
 
            SessionHolder.currentSession().getSess().flush();
 
            return StandartResults.SUCCESS;
 
        } catch (HibernateException ex) {
 
            SessionHolder.endSession();
 
            ex.printStackTrace();
 
            Messenger.addFatalMessage(ex.getMessage(),ex.getMessage());
 
            return StandardResults.FAIL;
 
        }
 
    }
   
 
}

This particular class was designed to be used in a JSF web application. Hence, the return types for methods are String's and there is a reference to StandardResults and Messenger class.
Messenger class is a generic class to present the user with a message.
StandardResults is what it's name stands for.

Please note, this may be used in JSF tables and lists only in places where the data is changed rarely or not concurrently.
But placing this element in a column in a h:dataTable will work as intended:




Also on your edit form page, you may use it without any potential problems, like this:


And on your new entry form page similarly:


In the end this technique saves a lot of time, by removing the need to create add, update and remove mehtods in your JSF managed-beans for those simple and boring CRUD operations.

* - For those who don't: see here or in simple terms this thing holds the session for the current thread and context - in the example, the call SessionHolder.currentSession() is translated to HibernateUtil.getSessionFactory().getCurrentSession().

How about a nice future for DBMS query languages?

Now thinking of SQL, it is a perfectly good language, but it is old. It is being pushed now to something that was not really intended for it.

We start needing to have more complex structures returned from our databases. See emergence of Amazon's SimpleDB, CouchDB and such.

What about a database where we talk to it in something more familiar? Even JavaScript seems nice... Well, the nice thing is that it's OOP model is basically tuples, the thing that most RDBMS'es understand perfectly well...
The problem at hand is the relation, but that is for another "pondering night".

Imagine this SQL query:

SELECT * FROM table_a WHERE id > 5 and name = 'Test'

In JavaScript form:

table_a.filter( {id:gt(5),name:"Test"} ).all()
(gt(5) here is a function returning a function that compares input so that it's greater than 5)

Readable? Not too much. Flexible? Definitely. Can it be improved? Absolutely!

There are more readable languages there... Like Python or Ruby.