Today, especially in the Pharma and Banking sectors, sooner or later you will be faced with the requirement of auditing. Detailed requirements will vary but usually at least tracking logons to the database is a must. Some companies need more information to pass their internal audits such as: Who created which objects, who fired which sql against the database, who was given which permissions and so on. In this post we’ll look at what PostgreSQL can offer here.
PostgreSQL comes with a comprehensive logging system by default. In my 9.5.4 instance there are 28 parameters related to logging:
(postgres@[local]:5438) [postgres] > select count(*) from pg_settings where name like 'log%';
count
-------
28
(1 row)
Not all of them are relevant when it comes to auditing but some can be used for a minimal auditing setup. For logons and loggoffs there are “log_connections” and “log_disconnections”:
(postgres@[local]:5438) [postgres] > alter system set log_connections=on;
ALTER SYSTEM
(postgres@[local]:5438) [postgres] > alter system set log_disconnections=on;
ALTER SYSTEM
(postgres@[local]:5438) [postgres] > select context from pg_settings where name in ('log_dicconnections','log_connections');
context
-------------------
superuser-backend
(1 row)
(postgres@[local]:5438) [postgres] > select pg_reload_conf();
pg_reload_conf
----------------
t
(1 row)
From now on, as soon as someone connects to or disconnects from the instance it is reported in the logfile:
2016-09-02 10:35:56.983 CEST - 2 - 13021 - [local] - postgres@postgres LOG: connection authorized: user=postgres database=postgres 2016-09-02 10:36:04.820 CEST - 3 - 13021 - [local] - postgres@postgres LOG: disconnection: session time: 0:00:07.837 user=postgres database=postgres host=[local]
Another parameter that might be useful for auditing is “log_statement”. When you set this to “ddl” all DDLs are logged, when you set it to “mod” all DDLs plus all statements that modify data will be logged. To log all statements there is the value of “all”.
(postgres@[local]:5438) [postgres] > alter system set log_statement='all'; ALTER SYSTEM
For new session all statements will be logged from now on:
2016-09-02 10:45:15.859 CEST - 3 - 13086 - [local] - postgres@postgres LOG: statement: create table t ( a int ); 2016-09-02 10:46:44.064 CEST - 4 - 13098 - [local] - postgres@postgres LOG: statement: insert into t values (1); 2016-09-02 10:47:00.162 CEST - 5 - 13098 - [local] - postgres@postgres LOG: statement: update t set a = 2; 2016-09-02 10:47:10.606 CEST - 6 - 13098 - [local] - postgres@postgres LOG: statement: delete from t; 2016-09-02 10:47:22.012 CEST - 7 - 13098 - [local] - postgres@postgres LOG: statement: truncate table t; 2016-09-02 10:47:25.284 CEST - 8 - 13098 - [local] - postgres@postgres LOG: statement: drop table t;
Be aware that your logfile can grow significantly if you turn this on and especially if you set the value to “all”.
That’s it more or less when it comes to auditing: You can audit logons, logoffs and SQL statements. This might be sufficient for your requirements but this also might not be sufficient for requirements. What do you do if you need e.g. to audit on object level? With the default logging parameters you can not do this. But, as always in PostgreSQL, there is an extension: pgaudit.
If you want to install this extension you’ll need the PostgreSQL source code. To show the complete procedure here is a PostgreSQL setup from source. Obiously the first step is to download and extract the source code:
postgres@pgbox:/u01/app/postgres/software/ [PG953] cd /u01/app/postgres/software/ postgres@pgbox:/u01/app/postgres/software/ [PG953] wget https://ftp.postgresql.org/pub/source/v9.5.4/postgresql-9.5.4.tar.bz2 --2016-09-02 09:39:29-- https://ftp.postgresql.org/pub/source/v9.5.4/postgresql-9.5.4.tar.bz2 Resolving ftp.postgresql.org (ftp.postgresql.org)... 213.189.17.228, 217.196.149.55, 87.238.57.227, ... Connecting to ftp.postgresql.org (ftp.postgresql.org)|213.189.17.228|:443... connected. HTTP request sent, awaiting response... 200 OK Length: 18496299 (18M) [application/x-bzip-compressed-tar] Saving to: ‘postgresql-9.5.4.tar.bz2’ 100%[==================================================================================>] 18'496'299 13.1MB/s in 1.3s 2016-09-02 09:39:30 (13.1 MB/s) - ‘postgresql-9.5.4.tar.bz2’ saved [18496299/18496299] postgres@pgbox:/u01/app/postgres/software/ [PG953] tar -axf postgresql-9.5.4.tar.bz2 postgres@pgbox:/u01/app/postgres/software/ [PG953] cd postgresql-9.5.4
Then do the usual configure, make and make install:
postgres@pgbox:/u01/app/postgres/software/ [PG953] PGHOME=/u01/app/postgres/product/95/db_4
postgres@pgbox:/u01/app/postgres/software/ [PG953] SEGSIZE=2
postgres@pgbox:/u01/app/postgres/software/ [PG953] BLOCKSIZE=8
postgres@pgbox:/u01/app/postgres/software/ [PG953] ./configure --prefix=${PGOME} \
--exec-prefix=${PGHOME} \
--bindir=${PGOME}/bin \
--libdir=${PGOME}/lib \
--sysconfdir=${PGOME}/etc \
--includedir=${PGOME}/include \
--datarootdir=${PGOME}/share \
--datadir=${PGOME}/share \
--with-pgport=5432 \
--with-perl \
--with-python \
--with-tcl \
--with-openssl \
--with-pam \
--with-ldap \
--with-libxml \
--with-libxslt \
--with-segsize=${SEGSIZE} \
--with-blocksize=${BLOCKSIZE} \
--with-wal-segsize=16 \
--with-extra-version=" dbi services build"
postgres@pgbox:/u01/app/postgres/software/postgresql-9.5.4/ [PG953] make world
postgres@pgbox:/u01/app/postgres/software/postgresql-9.5.4/ [PG953] make install
postgres@pgbox:/u01/app/postgres/software/postgresql-9.5.4/ [PG953] cd contrib
postgres@pgbox:/u01/app/postgres/software/postgresql-9.5.4/ [PG953] make install
Once this is done you can continue with the installation of the pgaudit extension:
postgres@pgbox:/u01/app/postgres/software/postgresql-9.5.4/ [PG953] git clone https://github.com/pgaudit/pgaudit.git postgres@pgbox:/u01/app/postgres/software/postgresql-9.5.4/contrib/ [PG953] cd pgaudit/ postgres@pgbox:/u01/app/postgres/software/postgresql-9.5.4/contrib/pgaudit/ [PG953] make -s check ============== creating temporary instance ============== ============== initializing database system ============== ============== starting postmaster ============== running on port 57736 with PID 8635 ============== creating database "contrib_regression" ============== CREATE DATABASE ALTER DATABASE ============== running regression test queries ============== test pgaudit ... ok ============== shutting down postmaster ============== ============== removing temporary instance ============== ===================== All 1 tests passed. ===================== postgres@pgbox:/u01/app/postgres/software/postgresql-9.5.4/contrib/pgaudit/ [PG953] make install /usr/bin/mkdir -p '/u01/app/postgres/product/95/db_4/lib' /usr/bin/mkdir -p '/u01/app/postgres/product/95/db_4/share/extension' /usr/bin/mkdir -p '/u01/app/postgres/product/95/db_4/share/extension' /usr/bin/install -c -m 755 pgaudit.so '/u01/app/postgres/product/95/db_4/lib/pgaudit.so' /usr/bin/install -c -m 644 ./pgaudit.control '/u01/app/postgres/product/95/db_4/share/extension/' /usr/bin/install -c -m 644 ./pgaudit--1.0.sql '/u01/app/postgres/product/95/db_4/share/extension/'
That’s it. Initialize a new cluster:
postgres@pgbox:/u01/app/postgres/software/ [PG954] initdb -D /u02/pgdata/PG954 -X /u03/pgdata/PG954
The files belonging to this database system will be owned by user "postgres".
This user must also own the server process.
The database cluster will be initialized with locales
COLLATE: en_US.UTF-8
CTYPE: en_US.UTF-8
MESSAGES: en_US.UTF-8
MONETARY: de_CH.UTF-8
NUMERIC: de_CH.UTF-8
TIME: en_US.UTF-8
The default database encoding has accordingly been set to "UTF8".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /u02/pgdata/PG954 ... ok
creating directory /u03/pgdata/PG954 ... ok
creating subdirectories ... ok
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting dynamic shared memory implementation ... posix
creating configuration files ... ok
creating template1 database in /u02/pgdata/PG954/base/1 ... ok
initializing pg_authid ... ok
initializing dependencies ... ok
creating system views ... ok
loading system objects' descriptions ... ok
creating collations ... ok
creating conversions ... ok
creating dictionaries ... ok
setting privileges on built-in objects ... ok
creating information schema ... ok
loading PL/pgSQL server-side language ... ok
vacuuming database template1 ... ok
copying template1 to template0 ... ok
copying template1 to postgres ... ok
syncing data to disk ... ok
WARNING: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
pg_ctl -D /u02/pgdata/PG954 -l logfile start
… and install the extension:
postgres@pgbox:/u02/pgdata/PG954/ [PG954] psql postgres
psql (9.5.4 dbi services build)
Type "help" for help.
(postgres@[local]:5438) [postgres] > create extension pgaudit;
ERROR: pgaudit must be loaded via shared_preload_libraries
Time: 2.226 ms
(postgres@[local]:5438) [postgres] > alter system set shared_preload_libraries='pgaudit';
ALTER SYSTEM
Time: 18.236 ms
##### Restart the PostgreSQL instance
(postgres@[local]:5438) [postgres] > show shared_preload_libraries ;
shared_preload_libraries
--------------------------
pgaudit
(1 row)
Time: 0.278 ms
(postgres@[local]:5438) [postgres] > create extension pgaudit;
CREATE EXTENSION
Time: 4.688 ms
(postgres@[local]:5438) [postgres] > \dx
List of installed extensions
Name | Version | Schema | Description
---------+---------+------------+---------------------------------
pgaudit | 1.0 | public | provides auditing functionality
plpgsql | 1.0 | pg_catalog | PL/pgSQL procedural language
(2 rows)
Ready. So, what can you do with it? As the documentation is quite well here are just a few examples.
To log all statements against a role:
(postgres@[local]:5438) [postgres] > alter system set pgaudit.log = 'ROLE';
Altering or creating roles from now on is reported in the logfile as:
2016-09-02 14:50:45.432 CEST - 9 - 13353 - [local] - postgres@postgres LOG: AUDIT: SESSION,2,1,ROLE,CREATE ROLE,,,create user uu login password , 2016-09-02 14:52:03.745 CEST - 16 - 13353 - [local] - postgres@postgres LOG: AUDIT: SESSION,3,1,ROLE,ALTER ROLE,,,alter user uu CREATEDB;, 2016-09-02 14:52:20.881 CEST - 18 - 13353 - [local] - postgres@postgres LOG: AUDIT: SESSION,4,1,ROLE,DROP ROLE,,,drop user uu;,
Object level auditing can be implemented like this (check the documentation for the meaning of the pgaudit.role parameter):
(postgres@[local]:5438) [postgres] > create user audit; CREATE ROLE (postgres@[local]:5438) [postgres] > create table taudit ( a int ); CREATE TABLE (postgres@[local]:5438) [postgres] > insert into taudit values ( 1 ); INSERT 0 1 (postgres@[local]:5438) [postgres] > grant select,delete on taudit to audit; GRANT (postgres@[local]:5438) [postgres] > alter system set pgaudit.role='audit'; ALTER SYSTEM (postgres@[local]:5438) [postgres] > select pg_reload_conf(); pg_reload_conf ---------------- t (1 row)
Once we touch the table:
(postgres@[local]:5438) [postgres] > select * from taudit; a --- 1 (1 row) (postgres@[local]:5438) [postgres] > update taudit set a = 4;
… the audit information appears in the logfile:
2016-09-02 14:57:10.198 CEST - 5 - 13708 - [local] - postgres@postgres LOG: AUDIT: OBJECT,1,1,READ,SELECT,TABLE,public.taudit,select * from taudit;, 2016-09-02 15:00:59.537 CEST - 9 - 13708 - [local] - postgres@postgres LOG: AUDIT: OBJECT,2,1,WRITE,UPDATE,TABLE,public.taudit,update taudit set a = 4;,
Have fun with auditing …
Cet article Auditing in PostgreSQL est apparu en premier sur Blog dbi services.
The difference is that you need a