|
Some applications need to look at
diverse data sets. This data probably already exists in your organization,
however it may not be accessible in a way that ties it to the other
organizations that may want or need to use it.
The reason for this is simple - organizations solved their business
problems by building point solutions. Often, these are customized very
specifically for the business need and this allows it to provide the best
possible data for that organization.
The problem is that there are often many instances of data that is related
to another instance of data, but these data stores cannot casually be merged
without substantially impacting the performance and usefulness of the other
systems. In reality, the issue is not typically one or 2 systems that need to
be tied together, but rather dozens of systems that have been created
specifically for the business need or highly customized software packages.
There was no business reason to make them compatible at the start, but
business models have changed and there is a need to bring this data together.
What are Meta Directories?
The concept of 'Federated' databases is
central to Meta Directory implementation. 'Federated' databases basically
allow you to join tables from Relational Databases that are distinct and
separate from each other. These are all SQL based and can be very complex to
setup and manage. These 'Federated' accesses normally are read only, because
the data that they tend to use is scattered within an organization and cannot
be 'locked' in any consistent fashion without causing major performance
problems.
A Meta Directory looks at this same issue, but approaches it in a slightly
different way. SQL is not the basis anymore, but the access is abstracted out
further to LDAP (Lightweight Directory Access Protocol) and/or XML (Extensible
Markup Language). The actual access still tries to do similar functions as an
SQL Based 'Federated' database, but no longer enforces SQL rules on the
transactions, opening the access up to any form of database, not just those
that can be configured to provide support for 'Federated' access.
Basically, a Meta Directory is a dynamic database access tool that can be
adapted to any existing set of databases to retrieve information and/or
provide translation services for the information returned. It should be able
to talk to practically any existing database that provides a TCP/IP access
point and be able to use it without requiring that the database, or the
applications that currently use it to be altered.
Typically a Meta Directory is a series of applications (usually called
'connectors' or 'listeners') that provide the services. the Services are
typically defined using Policies, where the Policies are stored within an LDAP
Directory server (Meta Directory Policy Store).
Back to top
Why use a Meta Directory
- Consolidating Data
Organizations often have multiple data stores, frequently scattered
across the country or the world and need a way to connect this information
together. They may also have a desire to get a better handle on the status
of existing production or resource utilization. These resources can be data
or they could be functions of any sort - for example, finding out the
thermostat settings in all of their buildings, or determining where someone
is today.
Many Meta Directory applications are used in Real-Time control
applications. Others are used in reporting. The most common use is to insert
a Meta Directory function in between applications, having the Meta Directory
application intercept the communications and act upon them. Most of the time
the data exists in a data store that the Meta Directory accesses. In some
instances, unique data will exist as a Meta Data within the Meta Directory
Policy Store. These Policies are used to implement business rules.
- When data needs be transformed to look consistent.
Any organization who can define the structure of the data they need to
resolve a specific business need, and identify that it needs to be the
result of data that appears in unique and seperate data sources, is a
candidate for Meta Directory applications.
Back to top
How are they Different than LDAP Directory Servers or Relational
Databases?
Most data stores are entities into themselves. Their data,
while accessible to the outside world thru various API's or applications, are
still bound as a point solution database. A Meta Directory is not really a
single place where data resides, rather its a method by which multiple
databases appear as a single data source.
Meta Directories don't care what the data is, only that they can access it
and return it as some part of a request for data. In this sense, the data that
it accesses is virtual, and can be from anything that can be connected to a
network.
Typically, a Meta Directory speaks any of LDAP, XML or SQL as needed. The
front end processing is most often done using LDAP or XML.
Back to top
Policies.
By default, most Meta Directory Applications are re-usable
for many different functions and until they have read their policies, they
usually don't do anything. The reason that they don't do anything is to
provide a level of security to their operations. Unless The Meta Directory
applications can identify their data sources and provide a security
infrastructure to access of that data, they should do practically nothing.
The Policies are read by the Meta Directory application at start-up as well
as at any interval that the application developers deem necessary.
More about Policies
Back to top
Configuration Management.
When you create a data map and define how a
Meta Directory application will support this, it should be committed to a safe
application management system.
The Meta Directory functions are highly dependent on their policy data and
will not operate as expected without it. Other than configuration specific
data within a Policy, this data should never be altered casually by any users.
Any changes to Policy must be done by Engineers who understand the various
systems and their access.
Benefits - Pros
- Since a Meta Directory application enables existing data, its ROI
appears very early on.
- Unlike data that is related can be tied to one-another. This allows
greater access and control of assets and resources.
- Data that is similar from different existing systems can operate as if
it was on a single unified data store. This alone often highlights
inconsistencies in the different systems and allows it to be addressed in a
controlled fashion - as a result, the data quality improves over time.
- Data Warehousing takes on a different perspective as Meta Directory
Application actions can be expanded very quickly by a very small staff. if
your business changes, your Meta Directory can adapt very quickly.
Issues - Cons
- The Meta Directory is sensitive to network availability - Its
functionality is directly tied to being able to access its data sources.
- Someone needs to take ownership of data issues. Inconsistencies will
appear once data is made visible; someone (organization) will need to
address this, otherwise the data will not be trusted.
- The Meta Directory applications will be highly distributed. Managing
this is a larger effort than point solution applications. Some one who
understands the whole system must be involved in resolving production
issues.
- Interfaces to existing systems must be well documented. This is not
always available for existing systems that the Meta Directory is attempting
to utilize. Minor uncoordinated changes in the point solution applications
may greatly affect the ability of the Meta Directory applications to resolve
data.
- Requires strong management backing - most point solution application
developers see Meta Directories as an invasion of their environment. Often
the biggest stumbling blocks are political rather than technical.
- Monitoring of applications that are used by the Meta Directory becomes
far more critical to operations. An outage in one area may bring the entire
Meta Functions down during that time. Some Meta Applications cache data for
these situations, however it is often impossible to cache all of the data
required for all operations. Redundant data stores may be required for some
applications, and this must be considered in any implementation.
Back to top
Summary
Meta Directory applications provide a service that is layered
upon your existing systems. Done correctly, it allows an organization to
access data in ways that previously were very hard to do and not adaptable to
changes in the business models.
Meta Directory solutions need not be expensive, or extensive. They are best
built out in controlled portions and adapted to organizational needs.
Typically, the communications and access methods need to be designed and
supported by the various data sources. Interface documents need to be created
and Service Level Agreements need to be hammered out to make sure that the
data that is required is there when its needed.
It all comes down to a simple fact, A Meta Directory becomes an
infrastructure solution. When its working correctly, most people won't even
know its there. Infrastructure is often hard to justify, because it becomes an
invisible operations layer. Once in place, it can invigorate, or if poorly
defined, it won't be used.
Any Meta Directory solution requires buy in from the highest levels of
management, otherwise it is destined for failure.
Back to top
Questions? Comments? Contact Engineering
|
