389 Directory Server - Recent changes [en]

Documentation

Rmeggins — Tue, 24 Nov 2009 19:40:26 GMT

/* Use and Deployment */

Revision as of 19:40, 24 November 2009		Current revision
Line 13:		Line 13:
	* [[FDS_Setup\|New Set up commands]] - Documentation for new 1.1 and later setup commands		* [[FDS_Setup\|New Set up commands]] - Documentation for new 1.1 and later setup commands
	* [[Install_Guide]] - Installation Guide		* [[Install_Guide]] - Installation Guide
-	* <font color="red">New!</font> [[DSGW_Install_Guide\|DSGW for 1.1 Installation Guide]]
-	* <font color="red">New!</font> [[DSGW_Building\|Building DSGW]]
-	* <font color="red">New!</font> [[DSML_Gateway\|DSML Gateway for 1.1 Docs]]
-	* <font color="red">New!</font> [[WebApps_Overview\|Using and configuring 389 web apps]], including the DSGW, DS Express, Org Chart, and Admin Express

	These are the official Red Hat Directory Server 8.1 docs but they apply to 389 DS 1.1:		These are the official Red Hat Directory Server 8.1 docs but they apply to 389 DS 1.1:
Line 35:		Line 31:
	* [[Performance Tuning]]		* [[Performance Tuning]]
	* [http://wiki.fedora-ve.org/Autenticaci%C3%B3n/FedoraDirectoryServer Fedora DS Documentation in Spanish (not official)]		* [http://wiki.fedora-ve.org/Autenticaci%C3%B3n/FedoraDirectoryServer Fedora DS Documentation in Spanish (not official)]
-		+	* [[DSGW_Install_Guide\|DSGW for 1.1 Installation Guide]]
		+	* [[DSGW_Building\|Building DSGW]]
		+	* [[DSML_Gateway\|DSML Gateway for 1.1 Docs]]
		+	* [[WebApps_Overview\|Using and configuring 389 web apps]], including the DSGW, DS Express, Org Chart, and Admin Express
	==== Quick Links ====		==== Quick Links ====

Line 136:		Line 135:
	====Design Docs====		====Design Docs====
	* [[Password_Syntax]] - Password syntax checking		* [[Password_Syntax]] - Password syntax checking
-	=====Proposed New Features=====
	* [[FHS_Packaging]] - Proposed Filesystem Hierarchy Standard Packaging layout		* [[FHS_Packaging]] - Proposed Filesystem Hierarchy Standard Packaging layout
-	* [[Account_Policy_Design]] - Account Management
	* [[Discrete_Packaging]] - Packaging 389 DS into smaller, discrete components		* [[Discrete_Packaging]] - Packaging 389 DS into smaller, discrete components
	* [[FDS_Into_FedoraCore]] - Work required to get 389 DS into Fedora Core		* [[FDS_Into_FedoraCore]] - Work required to get 389 DS into Fedora Core
	* [[Migration_From_10]] - How to migrate from a 1.0.x installation to 1.1 or later		* [[Migration_From_10]] - How to migrate from a 1.0.x installation to 1.1 or later
	* [[New_Setup_Design]] - New setup design for 1.1.0 and later		* [[New_Setup_Design]] - New setup design for 1.1.0 and later
		+	=====Proposed New Features=====
		+	* [[Account_Policy_Design]] - Account Management

	=====Current Features=====		=====Current Features=====

Named Pipe Log Script

Rmeggins — Tue, 24 Nov 2009 17:51:23 GMT

/* Starting pipe at server startup */

New page

The Named Pipe Log Script allows you to replace a log file with a named pipe attached to a script. The server can then send the log output to a script instead of to a log file. This allows you to do many different things such as:
* log only certain events e.g. failed binds, connections from certain ip addresses, etc.
* log only lines that match a certain pattern
* log only the last N lines - useful for enabling full error log debug levels in production environments
* send an email or other notification when a certain event is detected

The script is written in python, and allows plugins. By default, the script will log the last N lines (default 1000). There are two plugins provided - one to log only failed bind attempts, and one that will log only lines that match given regular expressions.

== Usage ==
ds-logpipe.py <name of pipe file to create or use> [options]
You must supply the filename of the named pipe. If it does not exist, the script will create it. If it exists and is a pipe, the script will use it. If it exists and is not a pipe, the script will abort.
-u|--user - userid to chown() the named pipe to - will affect files created by plugins too
-m|--maxlines - number of lines to keep in the circular buffer - default 1000
You can have the pipe script "monitor" another process, such as the directory server. You tell the pipe script the pid of the process, either directly, or by specifying the name of the file containing the pid. The pid file does not have to exist right away - the script will keep trying to read the file and check the pid until the timeout period is reached.
-s|--serverpidfile - name of file containing the server pid
-t|--servertimeout - number of seconds to wait for the pid file to exist and the pid to be running (default 60)
--serverpid - specify the pid of the server directly - the server must already be running
You can specify a plugin. The plugin must define a function that will be called with each line read from the pipe. You can optionally specify a pre function to be called when the plugin is loaded. Plugin command line arguments are passed to the pre function. You can optionally specify a post function to be called when the script exits.
--plugin=/path/to/pluginfile.py - the file must end in .py
pluginfile.arg1 ... pluginfile.argN
Each plugin may have arguments specified on the command line. If the plugin file is called pluginfile.py, the arguments are specified on the command line by '''pluginfile.argname'''. The argument names are whatever the plugin wants to use. The script will parse these and pass them to the plugin via the pre function, so the plugin must define a pre function in order to get the command line arguments. Multiple arguments with the same argname are passed as a list of values.

== Example ==
ds-logpipe.py /var/log/dirsrv/slapd-instance/errors.pipe -m 10000
Will create a named pipe which will keep the last 10000 lines read. You must configure the directory server to use errors.pipe as the error log instead of the default /var/log/dirsrv/slapd-instance/errors.
ds-logpipe.py /var/log/dirsrv/slapd-instance/errors.pipe \
--plugin=/usr/share/dirsrv/data/logregex.py logregex.regex="a bad error" > baderrors
Log only lines containing the string "a bad error" to the file baderrors.

== Directory Server Configuration ==
There are a couple of ways to configure the directory server to use the pipe:
=== Replace the default log file with the pipe ===
In this case, you will just replace the log file with the named pipe.
* Advantage - no server configuration required - however, if the server is running, you will have to tell it to reopen the log file
* Disadvantage - cannot use the log viewers (console, admin express) - other tools that expect the log to be a real file will fail (e.g. logconv.pl for the access log)

NOTE: You must move or delete the existing file. If you attempt to run the log pipe script and specify the name of an existing file, the script will abort if the file is not a named pipe. You may want to save the current file for safekeeping.
=== Create a new log pipe file ===
In this case, you will tell the script to create a new file (e.g. errors.pipe). You must configure the directory server to use the new log.
* Advantage - can use all of the log viewer and analysis tools
* Disadvantage - requires server configuration

=== Directory Server Log Configuration ===
Each of the 3 logs (access, error, audit) has several different configuration parameters. In order to enable the use of the named pipe for the log, you may want to modify them. For example, suppose you want to tell the server to use a pipe for the access log. You could use ldapmodify with the following LDIF:
dn: cn=config
changetype: modify
replace: nsslapd-accesslog-maxlogsperdir
nsslapd-accesslog-maxlogsperdir: 1
-
replace: nsslapd-accesslog-logexpirationtime
nsslapd-accesslog-logexpirationtime: -1
-
replace: nsslapd-accesslog-logrotationtime
nsslapd-accesslog-logrotationtime: -1
-
replace: nsslapd-accesslog
nsslapd-accesslog: /var/log/dirsrv/slapd-localhost/access.pipe
-
replace: nsslapd-accesslog-logbuffering
nsslapd-accesslog-logbuffering: off
* nsslapd-accesslog - specify the full path and filename of the named pipe - this case assumes the use of a new file and not the replacement of the existing file with the pipe
* nsslapd-accesslog-logbuffering - the script can do the buffering, so you usually want to turn this off
* nsslapd-accesslog-maxlogsperdir, nsslapd-accesslog-logexpirationtime, nsslapd-accesslog-logrotationtime - these control log rotation - if using a pipe, you do not want the server to rotate the log pipe
* nsslapd-accesslog-logging-enabled - by default the access log and error logs are enabled - the audit log is not - set this attribute to "on" or "off" to enable and disable logging

NOTE: Before doing this, you should save your current configuration so you can restore it later.
ldapsearch ... -s base -b "cn=config" nsslapd-accesslog-maxlogsperdir nsslapd-accesslog-logexpirationtime \
nsslapd-accesslog-logrotationtime nsslapd-accesslog nsslapd-accesslog > savedaccesslog.ldif

If the server is running, and the log pipe is active, if you use this LDIF with ldapmodify -f, the server will immediately close the current log and begin using the new one. This is a great way to debug a live running server.

The error log and audit log have similarly named configuration attributes e.g. nsslapd-errorlog, nsslapd-auditlog. Note that the audit log is disabled by default - use nsslapd-auditlog-logging-enabled: on to enable it.
== Starting pipe at server startup ==
You may want to start the pipe when the server starts up and stop the pipe when the server shuts down. That is, start using the pipe when the init script starts up the server, either at boot time or via the "service" command.

You must first configure the server to use the pipe (see above).

Next, modify the initconfig script for the server. This script is used by the init script to set parameters for each instance.
/etc/sysconfig/dirsrv-instancename
NOTE: DO NOT MODIFY /etc/sysconfig/dirsrv

The script will usually look something like this:
# These settings are used by the start-dirsrv and
# start-slapd scripts (as well as their associates stop
# and restart scripts). Do not edit them unless you know
# what you are doing.
SERVER_DIR=/usr/lib/dirsrv ; export SERVER_DIR
SERVERBIN_DIR=/usr/sbin ; export SERVERBIN_DIR
CONFIG_DIR=/etc/dirsrv/slapd-srv ; export CONFIG_DIR
INST_DIR=/usr/lib/dirsrv/slapd-srv ; export INST_DIR
RUN_DIR=/var/run/dirsrv ; export RUN_DIR
DS_ROOT= ; export DS_ROOT
PRODUCT_NAME=slapd ; export PRODUCT_NAME

# Put custom instance specific settings below here.
Do not change anything before that last line. After that last line, add something like the following:
# only keep the last 1000 lines of the error log
python /usr/bin/ds-logpipe.py /var/log/dirsrv/slapd-srv/errors.pipe -m 1000 -u nobody \
-s /var/run/dirsrv/slapd-srv.pid > /var/log/dirsrv/slapd-srv/errors &
# only log failed binds
python /usr/bin/ds-logpipe.py /var/log/dirsrv/slapd-srv/access.pipe -u nobody \
-s /var/run/dirsrv/slapd-srv.pid --plugin=/usr/share/dirsrv/data/failedbinds.py \
failedbinds.logfile=/var/log/dirsrv/slapd-srv/access.failedbinds &
Using the -s /var/run/dirsrv/slapd-srv.pid argument tells the scripts to exit when the server exits, and that the server will write its pid to the file /var/run/dirsrv/slapd-srv.pid when it starts up. If the server fails to start, the scripts will by default exit in 60 seconds. Use the -t parameter to specify a timeout other than 60 seconds.
== Writing Plugins ==
You can write your own plugin. Two plugins are provided with the directory server. Your plugin must define a function called '''plugin''', and may optionally define functions called '''pre''' and '''post'''.
=== plugin function ===
The plugin function is called with each line of the log.
def plugin(line):
retval = True
# do something with line
if something_is_bogus:
retval = False
return retval
line is the line from the log. It is a single line and will have the newline (\n) character at the end of the line. The plugin should normally return the value True. If the plugin detects some bad condition and wants to abort the pipe script, it should return the value False.
=== pre function ===
The pre function is called when the plugin is loaded and is passed the command line arguments for the plugin as a dict.
def pre(myargs):
retval = True
myarg = myargs['argname']
if isinstance(myarg, list): # handle list of values
else: # handle single value
if bad_problem:
retval = False
return retval
The command line arguments specified by '''pluginname.arg''' are passed into the pre function in the dict argument. The argument name with the ''pluginname.'' is the dict key, and the dict value for that key is the command line argument value. If there is more than one argument with the same name, they are converted to a list of values.
--plugin=/path/to/pluginname.py pluginname.arg1=foo pluginname.arg1=bar pluginname.arg2=baz
is converted to a dict like this:
{'arg1': ['foo', 'bar'],
'arg2': 'baz'}
=== post function ===
The post function is called when the log script is exiting.
def post(): # no arguments
# do something
# no return value

Roadmap

Rmeggins — Tue, 24 Nov 2009 15:26:42 GMT

/* Version 1.3 */

Revision as of 15:26, 24 November 2009		Current revision
Line 4:		Line 4:
	[[History]] contains the history of the project, and what features were added when.		[[History]] contains the history of the project, and what features were added when.

-	== Next Steps ==	+	=== Fedora Directory Server 1.2.0 - April 3, 2009 ===
-	=== Version 1.2 ===	+
-	The following features are planned for Fedora Directory Server 1.2. Each of the features listed here should be linked to a design document for that feature.	+
	* [[MemberOf_Plugin\|Automatically maintained memberOf attribute]]		* [[MemberOf_Plugin\|Automatically maintained memberOf attribute]]
	* [[LDAPI_and_AutoBind\|Server-side LDAPI support (LDAP over a UNIX domain socket)]]		* [[LDAPI_and_AutoBind\|Server-side LDAPI support (LDAP over a UNIX domain socket)]]
Line 17:		Line 15:
	* [[Dynamically_Reload_Schema\|Dynamically reload Schema via Task Interface]]		* [[Dynamically_Reload_Schema\|Dynamically reload Schema via Task Interface]]
	* [[64_bit_counters_design\|64-bit counters]]		* [[64_bit_counters_design\|64-bit counters]]
-	=== Version 1.2.1 ===	+
		+	=== 389 Directory Server 1.2.1 - August 17, 2009 ===
		+	NOTE: This is the first release that is branded as '''389'''. All of the RPMs have been marked as obsoleting their Fedora DS counterparts. When upgrading via yum, you must use yum '''upgrade''' (not update) so that the obsoletes will be processed.
		+
		+	NOTE: '''389-console''' is the command to run the console. This replaces fedora-idm-console.
		+
	* [[Linked_Attributes_Design\|Support links between two attributes]] (like memberOf but with other/configurable attributes)		* [[Linked_Attributes_Design\|Support links between two attributes]] (like memberOf but with other/configurable attributes)
	* Support dereferencing control - http://www.openldap.org/devel/cvsweb.cgi/~checkout~/doc/drafts/draft-masarati-ldap-deref-xx.txt		* Support dereferencing control - http://www.openldap.org/devel/cvsweb.cgi/~checkout~/doc/drafts/draft-masarati-ldap-deref-xx.txt
Line 44:		Line 47:
	** the SASL I/O layer has been reworked as a push-able NSPR I/O layer		** the SASL I/O layer has been reworked as a push-able NSPR I/O layer

		+	=== 389 Directory Server 1.2.2 - August 26, 2009 ===
		+	This was a bug fix release - no new features.
		+
		+	=== 389 Directory Server 1.2.3 - October 7, 2009 ===
		+	* Ability to set resource limits (sizelimit, timelimit, look through limit) specifically for anonymous connections
		+	** This is useful when you want to have different limits for regular users and anonymous users
		+	** Set the attribute '''nsslapd-anonlimitsdn''' in cn=config to the DN of the entry that you want to use as the "template" entry. This is a dummy entry that you have to create. Then you set whatever resource limits you want to apply to anonymous to that dummy entry, and those limits will apply to anonymous users.
		+	* Access based on the security strength of the connection
		+	** There is a new ACI keyword - '''minssf''' - this allows you to set access control based on how secure the connection is
		+	** There is a global server setting in cn=config - '''nsslapd-minssf''' - that allows you to reject operations based on how secure the connection is
		+	* Ability to shut off anonymous access
		+	** This adds a new config switch in cn=config - '''nsslapd-allow-anonymous-access''' - that allows one to restrict all anonymous access. When this is enabled, the connection dispatch code will only allow BIND operations through for an unauthenticated user. The BIND code will only allow the operation through if it's not an anonymous or unauthenticated BIND.
		+	=== 389 Directory Server 1.2.4 - November 4, 2009 ===
		+	* Support for Salted MD5 (SMD5) hashes. These are supported for migration purposes only. You should not use SMD5 for new passwords - use SSHA256
		+
		+	== Next Steps ==
	=== Version 1.3 ===		=== Version 1.3 ===
-	The following features are planned for 389 Directory Server 1.3. Not all of these features will actually end up in 1.3, but these are the features we are considering. Each of the features listed here should be linked to a design document for that feature.	+	The following features are planned for 389 Directory Server 1.3.
		+
		+	* Subtree Rename and Entry Move (modifyDN with newSuperior)
		+	** https://bugzilla.redhat.com/show_bug.cgi?id=429005
		+	** ability to rename a node that has children
		+	** ability to move a node, with or without children, to another parent node
		+
		+	* Security Enhancements
		+	** [[SELinux_Policy\|SELinux Policy]]
		+	*** https://bugzilla.redhat.com/show_bug.cgi?id=442228
		+	*** including all sub-components such as Admin server, dsgw and DS console if applicable
		+
		+	* [[Named Pipe Log Script\|Named Pipe Log Script]]
		+	** provide script which allows you to replace one or all log files with a named pipe script to do circular buffering, filtering, notifications, etc.
		+
		+	* Auto-generated SLAPI reference
		+	** generate plugin programmers reference from code + comments (e.g. doxygen)
		+
		+	* [[Plugin_Ordering\|Plug-in Ordering]]
		+	** Allow the order that plug-ins are invoked in to be defined.
		+
		+	=== Future ===
	* [[Use_OpenLDAP_Clients_In_389\|Support OpenLDAP client libs]] - allow the use of OpenLDAP client libs in addition to mozldap		* [[Use_OpenLDAP_Clients_In_389\|Support OpenLDAP client libs]] - allow the use of OpenLDAP client libs in addition to mozldap
	** Includes support for NSS - see http://www.openldap.org/its/index.cgi/Contrib?id=5696		** Includes support for NSS - see http://www.openldap.org/its/index.cgi/Contrib?id=5696
Line 55:		Line 95:
	** Need ability to specify and ordering for attribute values or the ability to retrieve attributes values in a certain order - control? attr subtype?		** Need ability to specify and ordering for attribute values or the ability to retrieve attributes values in a certain order - control? attr subtype?
	*** This may be needed less if the painful tool (Microsoft Managment console) is not used.		*** This may be needed less if the painful tool (Microsoft Managment console) is not used.
-	* Subtree Rename and Entry Move (modifyDN with newSuperior)
-	** https://bugzilla.redhat.com/show_bug.cgi?id=429005
-	** ability to rename a node that has children
-	** ability to move a node, with or without children, to another parent node
	* Schema		* Schema
	** improve parser, fix long standing bugs		** improve parser, fix long standing bugs
	** possibly support openldap style schema files		** possibly support openldap style schema files
-	* Security Enhancements
-	** [[SELinux_Policy\|SELinux Policy]]
-	*** https://bugzilla.redhat.com/show_bug.cgi?id=442228
-	*** including all sub-components such as Admin server, dsgw and DS console if applicable
-	** [[Anonymous_Access_Switch\|Anonymous Access Switch]]
-	***Option to disable all anonymous access (1.2.0 can disable unauthenticated BIND operations)
-	** [[Anonymous_Resource_Limits\|Anonymous Resource Limits]]
-	** [[Require_Secure_Binds_switch\|Require Secure Binds Switch]]
-	*** https://bugzilla.redhat.com/show_bug.cgi?id=454496
-	*** avoid clear text password sent over unsecure connection
-	*** 1.2.1 has support for requiring a secure connection for simple bind
-	** [[SSF_Restrictions\|SSF Restrictions]]
-	*** minimum SSF setting and ssf bind rule for access control
	** Option to turn off SASL mechanisms		** Option to turn off SASL mechanisms
	*** https://bugzilla.redhat.com/show_bug.cgi?id=206053		*** https://bugzilla.redhat.com/show_bug.cgi?id=206053
Line 80:		Line 103:
	** Password Policy		** Password Policy
	*** Make the DS password policy pluggable - be able to support plugins such as cracklib		*** Make the DS password policy pluggable - be able to support plugins such as cracklib
-	** Access Logging
-	*** provide script which allows you to replace one or all log files with a named pipe script to do circular buffering, filtering, notifications, etc.
-	** SMD5 (salted MD5) password storage hash
-	*** https://bugzilla.redhat.com/show_bug.cgi?id=221905
-	*** primarily for migration and interoperability
	** Restrict directory manager bind by host/IP address		** Restrict directory manager bind by host/IP address
	*** https://bugzilla.redhat.com/show_bug.cgi?id=458187		*** https://bugzilla.redhat.com/show_bug.cgi?id=458187
	*** e.g. only allow root dn to bind from 127.0.0.1 - turn off remote root dn bind		*** e.g. only allow root dn to bind from 127.0.0.1 - turn off remote root dn bind
	** Get rid of certmap.conf - use SASL mapping (cert auth is really just SASL/EXTERNAL)		** Get rid of certmap.conf - use SASL mapping (cert auth is really just SASL/EXTERNAL)
-	* Auto-generated SLAPI reference
-	** generate plugin programmers reference from code + comments (e.g. doxygen)
	* Support additional standard attribute syntaxes, matching rules		* Support additional standard attribute syntaxes, matching rules
	** https://bugzilla.redhat.com/show_bug.cgi?id=479753		** https://bugzilla.redhat.com/show_bug.cgi?id=479753
Line 101:		Line 117:
	* Move changelog into main database environment		* Move changelog into main database environment
	** Improved performance, robustness and consistency		** Improved performance, robustness and consistency
-	* [[Plugin_Ordering\|Plug-in Ordering]]
-	** Allow the order that plug-ins are invoked in to be defined.
	* Support GUIDs in ACIs to reference entries		* Support GUIDs in ACIs to reference entries
	** confirm group filter functionality in 1.2 supports this		** confirm group filter functionality in 1.2 supports this