Format Date in Java using ISO8601 including timezone

To format a java.util.Date as an ISO8601 timestamp including the correct timezone, java.time.OffsetDateTime can be used.

package de.lhorn.playground;

import java.time.Instant;
import java.time.OffsetDateTime;
import java.time.ZoneOffset;
import java.util.Date;
import java.util.TimeZone;

public class FormatDatetimeExample {

	public static void main(String[] args) {
		// The Date we want to format.
		Date date = new Date();
		long millis = date.getTime();

		// The local timezone of the system we run this code on.
		TimeZone timeZone = TimeZone.getDefault();

		// We calculate the timezone offset that applies to the given Date.
		int offset = timeZone.getOffset(millis);
		ZoneOffset zoneOffset = ZoneOffset.ofTotalSeconds(offset / 1_000);

		// Convert the Date into an OffsetDateTime using the calculated ZoneOffset.
		Instant instant = date.toInstant();
		OffsetDateTime odt = instant.atOffset(zoneOffset);

		// The output is something like "2019-10-30T08:16:39.535+01:00" which
		// includes the correct timezone in which I am currently located (Europe/Berlin).
		System.out.println("odt: " + odt.toString());
	}
}
2019-10-30

Mirror of NVD database

For development of Java services I use the recommended Maven plugin dependency-check-maven to scan for vulerable dependencies. Since the database used by this plugin is down from time to time, a local mirror of the database can be created using nist-data-mirror.

I've created such a mirror which you can use. Edit the <configuration> section of the plugin:

<configuration>
  <cveUrlModified>https://www.lhorn.de/nist/nvdcve-1.0-modified.json.gz</cveUrlModified>
  <cveUrlBase>https://www.lhorn.de/nist/nvdcve-1.0-%d.json.gz</cveUrlBase>
</configuration>

The mirror is refreshed daily at 0437 CE(S)T.

2019-10-01

HOWTO use JWT in Go

package main

import (
    "errors"
    "fmt"
    "io/ioutil"

    jwt "github.com/dgrijalva/jwt-go"
    "github.com/satori/go.uuid"
)

func keyfunc(token *jwt.Token) (interface{}, error) {
    kid := token.Header["kid"]
    str, ok := kid.(string)
    if !ok {
        return nil, errors.New("failed to get kid from token header")
    }
    pubPEM, err := ioutil.ReadFile(string(str))
    if err != nil {
        return nil, err
    }
    pubParsed, err := jwt.ParseRSAPublicKeyFromPEM(pubPEM)
    if err != nil {
        return nil, err
    }

    return pubParsed, nil
}

type sbcsClaims struct {
    Upn    string   `json:"upn"`
    Groups []string `json:"groups"`
    jwt.StandardClaims
}

func main() {

    // Private key.
    privPEM, err := ioutil.ReadFile("key.pem")
    if err != nil {
        panic(err)
    }
    privParsed, err := jwt.ParseRSAPrivateKeyFromPEM(privPEM)
    if err != nil {
        panic(err)
    }

    // Create JWT.
    stdClaims := jwt.StandardClaims{
        Issuer:    "Go Ticketserver 1.0",
        Id:        uuid.NewV4().String(),
        Subject:   "john.doe",
        ExpiresAt: 1521422680,
    }
    claims := sbcsClaims{
        Upn:    "john.doe@example.com",
        Groups: []string{"GROUP1", "GROUP2"},
    }
    claims.StandardClaims = stdClaims

    token := jwt.NewWithClaims(jwt.SigningMethodRS256, claims)
    token.Header["kid"] = "key.pub.pem"
    signed, err := token.SignedString(privParsed)
    if err != nil {
        panic(err)
    }
    fmt.Println(signed)

    fmt.Println("====")

    // Validate JWT.
    parsedToken, err := jwt.Parse(signed, keyfunc)
    if err != nil {
        fmt.Println("token is invalid: ", err)
    } else {
        fmt.Println(parsedToken.Claims)
    }

}
2019-06-16

Datetime with zone for PostgreSQL

A timestamp should be saved in two columns, one containing the local timestamp, one the name of the zone. Queries on these columns require casts which make usage of an index on an expression advisable.

drop table if exists t;

-- Create temporary table with local datetime and time zone.
create temporary table t (
    id SERIAL primary key,
    "datetime" timestamp without time zone not null default now(),
    "zone" varchar(255) not null default 'Europe/Berlin'
);

-- Insert 262,800 generated rows.
insert into t (
    "datetime",
    "zone"
)
select 
    "datetime",
    'Europe/Berlin'
from
    generate_series('2018-01-01T00:00:01'::timestamp, '2018-12-31T23:59:59'::timestamp, '2 minutes') as "datetime";

-- Index on the "datetime" only.
create index on t ("datetime", id);
analyze;

-- This index is used.
explain
select
    "datetime",
    "zone",
    id
from
    t
where
    "datetime" >= '2018-11-02T09:00:00'
and
    "datetime" < '2018-11-02T10:00:00'
;

/*
+------------------------------------------------------------------------------------------------------------------------------------------------------------+
| QUERY PLAN                                                                                                                                                 |
|------------------------------------------------------------------------------------------------------------------------------------------------------------|
| Bitmap Heap Scan on t  (cost=37.89..1885.03 rows=1314 width=528)                                                                                           |
|   Recheck Cond: ((datetime >= '2018-11-02 09:00:00'::timestamp without time zone) AND (datetime < '2018-11-01 10:00:00'::timestamp without time zone))     |
|   ->  Bitmap Index Scan on t_datetime_id_idx  (cost=0.00..37.56 rows=1314 width=0)                                                                         |
|         Index Cond: ((datetime >= '2018-11-02 09:00:00'::timestamp without time zone) AND (datetime < '2018-11-01 10:00:00'::timestamp without time zone)) |
+------------------------------------------------------------------------------------------------------------------------------------------------------------+
*/

-- Converting the local datetime to UTC using the stored "zone", the index is not used.
explain
select
    "datetime",
    "zone",
    id 
from
    t
where
    "datetime" at time zone "zone" at time zone 'UTC' >= '2018-11-02T09:00:00'
and
    "datetime" at time zone "zone" at time zone 'UTC' < '2018-11-02T10:00:00'
;

/*
+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
| QUERY PLAN
|-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
| Seq Scan on t  (cost=0.00..8503.00 rows=1314 width=528)
|   Filter: ((timezone('UTC'::text, timezone((zone)::text, datetime)) >= '2018-11-02 09:00:00'::timestamp without time zone) AND (timezone('UTC'::text, timezone((zone)::text, datetime)) < '2018-11-01 10:00:00'::t
+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
*/

-- Add an index on the used expression.
create index on t (timezone('UTC'::text, timezone((zone)::text, datetime)));
analyze;

-- This index is used.
explain
select
    "datetime",
    "zone",
    id 
from
    t
where
    "datetime" at time zone "zone" at time zone 'UTC' >= '2018-11-02T09:00:00'
and
    "datetime" at time zone "zone" at time zone 'UTC' < '2018-11-02T10:00:00'
;

/*
CREATE INDEX
+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
| QUERY PLAN                                                                                                                                                                                                        
|-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
| Bitmap Heap Scan on t  (cost=29.89..1890.17 rows=1314 width=528)                                                                                                                                                  
|   Recheck Cond: ((timezone('UTC'::text, timezone((zone)::text, datetime)) >= '2018-11-02 09:00:00'::timestamp without time zone) AND (timezone('UTC'::text, timezone((zone)::text, datetime)) < '2018-11-01 10:00:
|   ->  Bitmap Index Scan on t_timezone_idx  (cost=0.00..29.56 rows=1314 width=0)                                                                                                                                   
|         Index Cond: ((timezone('UTC'::text, timezone((zone)::text, datetime)) >= '2018-11-02 09:00:00'::timestamp without time zone) AND (timezone('UTC'::text, timezone((zone)::text, datetime)) < '2018-11-01 10
+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
*/
2019-06-16

Python function to get the PID of a named process

This Python function will return the PID of a namen Linix process.

import os
import subprocess

def getPid(name):
    """Get the process ID (pid) of a named process."""
    ps = subprocess.run(["ps", "cax"], stdout=subprocess.PIPE, stderr=subprocess.PIPE)
    if ps.returncode == 0:
        for line in ps.stdout.decode("UTF-8").split("\n"):
            if name in line:
                pid = line.split()[0]
                return pid
    else:
        print(ps.stderr, file=os.stderr)
        return None


print(getPid("firefox"))
print(getPid("eclipse"))
print(getPid("foo"))
2019-06-16

HOWTO get notifications about new Wikipedia pages

Wikipedia provides SSE acces to all changes made. It is easy to consume this event stream using Python.

import json
from sseclient import SSEClient as EventSource

# The SSE URL.
url = 'https://stream.wikimedia.org/v2/stream/recentchange'

# Loop over the events forever.
for event in EventSource(url):
    if event.event == 'message':
        try:
            change = json.loads(event.data)
            
            # If the change was not made by a bot and it was made on de.wikipedia.org,
            # print title, URL and change comment. More details are available in the
            # 'change' object.
            if not change["bot"] and change["meta"]["domain"] == "de.wikipedia.org":
                print("=" * 20)
                print(change["title"] + " <" + change["meta"]["uri"] + ">")
                print(change["comment"])
        except ValueError:
            pass
2019-06-14

HOWTO use the Alfresco CMIS Browser Binding

The CMIS bindings of Alfresco are not very well documented. This is what I've found out.

Usage of the various encodings (using the curl -F and -d options) can probably be improved.

Get Node Metadata

$ curl -u 'admin:secret'\
http://host/alfresco/api/-default-/public/cmis/versions/1.1/browser/root?objectId=89098071-2422-4b13-ad34-655e8cc17921&cmisselector=object'

Note that it is possible to query directly on the root. Together with the objectId query parameter, this is enough to address the node.

The query parameter cmisselector must have the value object to get the node metadata.

This query returns application/json.

Get Node Content

$ curl -u 'admin:secret' \
'http://host/alfresco/api/-default-/public/cmis/versions/1.1/browser/root?objectId=89098071-2422-4b13-ad34-655e8cc17921'

Without the cmisselector query parameter, the node content is returned.

Search for Node by objectId

It is also possible to search for a node using a CMIS query.

$ curl -u 'admin:secret' \
-F cmisaction=query \
-F "statement=SELECT * FROM cmis:document WHERE cmis:objectId = '89098071-2422-4b13-ad34-655e8cc17921'" \
http://host/alfresco/api/-default-/public/cmis/versions/1.1/browser

There are two form parameters, cmisaction must have the value query, statement contains the CMIS query that matches the node with the objectId of the node.

Note that the response has a different structure: It is a JSON array with one element; only the properties specified in the SELECT are returned. If you want properties that are defined on a custom type or on aspects, you must use a CMIS JOIN to include them.

Search for Nodes

A similar request can be used to search for nodes.

$ curl -u 'admin:secret' \
-F cmisaction=query \
-F "statement=SELECT * FROM my:CustomType AS ct JOIN your:CustomAspect AS ca ON ct.cmis:objectId = ca.cmis:objectId WHERE ca.your:customProperty = 'some value' ORDER BY ct.my:sortProperty DESC" \
-F skipCount=0 \
-F maxItems=10 \
http://host/alfresco/api/-default-/public/cmis/versions/1.1/browser

This is very similar to the previous one, only the CMIS SELECT is different. It uses a JOIN to include properties of a custom type and a custom aspect. Without this, only the default CMIS properties would be included in the result and it would not be possible to to query on a custom property.

Note that aliases (ct and ca) are used in the FROM and in the WHERE condition. The join must be done ON ct.cmis:objectId = ca.cmis:objectId.

CMIS supports ORDER BY in the SELECT and skipCount and maxItems parameters.

Update Node Properties

$ curl -k -u 'admin:secret' \
-d 'cmisaction=update&propertyId[0]=my:customProperty&propertyValue[0]=some value' \
'http://host/alfresco/api/-default-/public/cmis/versions/1.1/browser/root?objectId=89098071-2422-4b13-ad34-655e8cc17921'

This time the cmisaction has the value update. The body also includes one pair of propertyId and propertyValue using an array notation.

Change Node Content

TODO

Create New Node

TODO

2017-12-12