Categories
Miscellaneous PHP

Accessing a Vault in PHP

I admit it. I was “hacked”. It was not really a hack, it was my mistake not to notice that one of my deployment logs was accessible for everyone to read and – exceptionally for a specific test performed back then – revealing configuration data including a password to an AWS service. The leakage has been fixed and the password changed. So, you don’t need to search for it anymore ๐Ÿ™‚

However, the incident was reason enough for me to further secure my applications and introduce a vault for secret information. As most of my applications are based on PHP, I tried to find some ready-to-use code (and found a few). But all these libraries and SDKs are very heavyweight as they address much more use cases than just accessing a vault in order to fetch a secret read-only.

So I wrote a lightweight version of a PHP vault that not only accesses a Hashicorp Vault but also provides an abstract API so that my applications do not need to know what vault is being used or even where the secrets are stored.

So here is a code snippet that demonstrates how to use it with a Hashicorp Vault:

// Create configuration
$config = array(
	'type'   => 'hashicorp',
	'config' => array(
		'uri'      => 'https://127.0.0.1:8200/v1',
		'roleId'   => '<app-role-id>',
		'secretId' => '<secret-id>'
	)
);

// Create the vault instance
try {
	$vault = \Vault\VaultFactory::create($config);
} catch (\Vault\VaultException $e) {
	// Vault could not be created
}

With that vault, I can now access my secrets transparently within application code:

try {
	$mySecret = $vault->get('my/secret");
	$username = $mySecret->get('username');
	$password = $mySecret->get('password');
} catch (\Vault\VaultException $e) {
	// secret was not found
}

I even can further abstract this by not even knowing that there is a vault involved:

$callback = new \Vault\CredentialsProvider($vault, 'my/secret');

try {
	$username = $callback->getUsername();
	$password = $callback->getPassword();
} catch (\Vault\VaultException $e) {
	// Secret cannot be retrieved or does not exist
}

I can now integrate this pattern in all my PHP projects without immediately putting the secrets in a Hashicorp vault. The framework already comes with vault implementations that are based on configuration files or objects.

All code is publicly available at GitHub for reuse. The documentation there gives more code examples on how to use other vaults, e.g. when you want to start slowly and only manage your secrets in a configuration file.

Categories
Kubernetes

Kubernetes Service names in HELM templates

Based on my previous post, here comes a snippet that will correctly produce a full DNS name of a service in the cluster from the same namespace.

{{/*
Makes a full hostname from the given string if it's not one already or an IP address.
Attaches ".<namespace>.svc.cluster.local" to the end and includes the release name if required.
Please note that you need to call this template with (dict "Context" . "Value" "your-value")
*/}}
{{- define "prefix.serviceName" -}}
{{- if include "prefix.isIpAddress" .Value }}
    {{- print .Value }}
{{- else -}}
    {{- $parts := splitList "." .Value -}}
    {{- if gt (len $parts) 1 -}}
        {{- print .Value }}
    {{- else -}}
        {{- if eq .Context.Chart.Name .Context.Release.Name -}}
            {{- printf "%s.%s.svc.cluster.local" .Value .Context.Release.Namespace }}
        {{- else -}}
            {{- printf "%s-%s.%s.svc.cluster.local" .Context.Release.Name .Value .Context.Release.Namespace }}
        {{- end -}}

    {{- end -}}
{{- end -}}
{{- end -}}

Please note that using the template is a bit more cumbersome due to some Go language issues:

serviceName-anIpAddress:  {{ include "prefix.serviceName" (dict "Context" . "Value" "1.2.3.4") }}
serviceName-anIpAddress2: {{ include "prefix.serviceName" (dict "Context" . "Value" "1.0.3.4") }}
serviceName-NoIpAddress:  {{ include "prefix.serviceName" (dict "Context" . "Value" "1.2.3.4.5") }}
serviceName-NoIpAddress2: {{ include "prefix.serviceName" (dict "Context" . "Value" "hello") }}
serviceName-NoIpAddress3: {{ include "prefix.serviceName" (dict "Context" . "Value" "hello.svc") }}
serviceName-NoIpAddress4: {{ include "prefix.serviceName" (dict "Context" . "Value" "hello.svc.cluster.local") }}
serviceName-NoIpAddress5: {{ include "prefix.serviceName" (dict "Context" . "Value" "1") }}

The template needs access to the root context. So the dict function is used to pass the context and the actual, simple service name.

Feel free to adjust the function when you need another namespace as an argument.

Categories
Kubernetes

HELM template to detect IP address

I was in a need to detect whether the content of a variable is an IP address or not. I guess the function is not perfect, but it fulfills the basic need:

{{/*
Test if the given value is an IP address
*/}}
{{- define "prefix.isIpAddress" -}}
{{- $rc := . -}}
{{- $parts := splitList "." . -}}
{{- if eq (len $parts) 4 -}}
    {{- range $parts -}}
        {{- if and (not (atoi .)) (ne . "0") -}}
            {{- $rc = "" -}}
        {{- end -}}
    {{- end -}}
{{- else -}}
    {{- $rc = "" -}}
{{- end -}}
{{- print $rc }}
{{- end -}}

The function at least detects these values correctly:

{{ include "prefix.isIpAddress" "1.2.3.4" }}
{{ include "prefix.isIpAddress" "1.0.3.4" }}
{{ include "prefix.isIpAddress" "1.2.3.4.5" }}
{{ include "prefix.isIpAddress" "hello" }}
{{ include "prefix.isIpAddress" "hello.svc" }}
{{ include "prefix.isIpAddress" "hello.svc.tld.com" }}
Categories
Linux Miscellaneous Shell

Using sendmail with a Relay Host

It is useful when Docker containers can send e-mail to you in case there is an error condition that needs attention. Here is how to.

Install the sendmail package from your distribution and edit the file /etc/mail/sendmail.mc. Add this line to the end of it:

define('SMART_HOST', '<dns-name-of-your-relay')

Done! Just restart sendmail:

/etc/init.d/sendmail stop
/etc/init.d/sendmail start
Categories
Bugzilla for Java

What is the current status of Bugzilla?

I am honest. I am confused about the current status of Bugzilla. My personal environment does not list anyone or any project anymore that uses the most famous bug tracker software – for years. And this situation now seems to reflect in the current development activities in Bugzilla’s official GitHub repository. The last commit listed as of today was on February 1st, almost 3 months ago. Even worse: only 10 commits are logged within the last 12 months – on their active 5.2 branch. The last official release of Bugzilla was published even before that: on February 9th, 2019.

So it appears to me that the project dies a slow death – caused by the community that has migrated away to other bug trackers. And as sad as this seems, it is not a very surprising fact. Bugzilla was ignoring the needs of their users for too long. I remember a privacy issue to hide e-mail addresses from public that is still open (after many years). Bugzilla has become a dinosaur in a modern world. It is based on Perl, hard to setup on modern systems and misses many of the flexibility that other trackers integrated much faster.

Having said this, I am convinced now – more than ever – that my personal project B4J – Bugzilla for Java will not be maintained anymore. I will most likely fix bugs. But there will be no enhancement anymore.

Good-bye old friend, Bugzilla!

Categories
Bugzilla for Java CSV Java RS Library

Several new Software Releases available

As mentioned in previous post, most of my projects have been touched lately. So here is a list of the latest releases that you might find useful to integrate:

Also, there is a new project: EventBroker – a MQTT-alike event broker to enable losely coupled microservices being synchronized. It is written in Java and runs as docker container. Check it out when you look for an easy way to signal events to REST services in your distributed environment. Fun Feature: a special subscription topic for timer events that will enable you to implement cron jobs inside microservices – triggered by a REST call.

Categories
Miscellaneous

Long Time No See

A long time has passed since the last post on this blog. Not because I was lazy. It was merely because there were more important things to do than writing blog posts about things that most people can look-up in the internet anyway.

However, the Open Source software projects were still going on. Not so frequent updates but once in a while. The current Corona pandemic now gives me some possibilities to finish things that were long time on my list. First and most importantly is to gain independance of hosting all my software on my own and maintaining the infrastructure for it. Still, some main parts will be on me. Such as build tools, issue tracking and automation.

However, I managed to host all my code now at GitHub. This task alone cost me about two weeks until each and every Subversion repository was migrated. I have been writing code now for more than 20 years. That’s why about 110 software projects piled up at my previous Subversion repository. Most of them are not public, only 26 can be accessed by everyone. But migrating all 110 physically took me 3 days. Another 10 days I was busy to update the CI/CD pipelines for the still active projects (around 50). And the last week passed with upgrading the Open Source projects to new software versions, documenting them, changing the workflows, upgrading build tools and writing CI/CD tools for these changes. Finally, I managed to bump up the versions of the major OSS projects – after 3 weeks of work. Most of them were API breaking. That’s why the major versions increased (Check Maven Central for an overview).

You will find updates on them here in this blog – and you will see more updates coming soon. The main changes are:

  • Upgrading to Java 9: My Java projects will not support any older runtime environment.
  • Documentation moves to GitHub along withe code and the respective version. It is still going on. So this blog will become less important for documentation and the respective sections will be removed from the menu (but still be available).
  • Development workflow will follow the Gitflow workflow model now.

Feel free to contact me for any of the projects, the new or the old ones. For the moment, I wish you all the best and stay healthy!

Ralph

PS: Of course, I will try to blog more IT stuff and more frequently than before ๐Ÿ™‚

Categories
Kubernetes

IPv6 with Kubernetes

Awwww – so much work I had put into setting up a Kubernetes cluster (this blog will run there in a few days). I set up the pods and containers, cron jobs, services, and, and, and. Then I started renewing my SSL certificates from LetsEncrypt. This renewal failed hilariously, but with a weird error message:

1
Timeout

What? I can reach my websites. Did I miss something? I checked connectivity. The IP addresses were right, the ACME challenge directory was available as required by LetsEncrypt, the DNS was working properly. Why couldn’t LetsEncrypt servers not reach my cluster? I soon found out that they prefer IPv6 over IPv4 which I had both enabled. But the IPv6 connection failed. From everywhere. Ping6 though succeeded.

Further analysis revealed that Kubernetes is not able to expose IPv6 services at all (or at least at now, so I researched). What shall I do now? All my work was based on the assumption that IPv4 and IPv6 will be there. But it’s not with Kubernetes. Of course I could move my reverse proxy out of Kubernetes and put it in front of it. But that would require more work as all the automation scripts for LetsEncrypt would need to be rebased. Testing again and again. Let aside the disadvantage of not having it all self-contained in containers anymore. Another solution must be there.

Luckily there was an easy solution: socat. It’s a small Linux tool that can copy network traffic from one socket to another. So that was setup easily with a systemd script (sock_80.service):

1
2
3
4
5
6
7
8
9
10
11
12
[Unit]
 Description=socat Service 80
 After=network.target
 
[Service]
 Type=simple
 User=root
 ExecStart=/usr/bin/socat -lf /var/log/socat80.log TCP6-LISTEN:80,reuseaddr,fork,bind=[ip6-address-goes-here] TCP4:ip4-address-goes-here:80
 Restart=on-abort
 
[Install]
 WantedBy=multi-user.target

That’s it. Enabled it (systemctl enable sock_80.service), reloaded systemd (systemctl daemon-reload), and started the service (systemctl start sock_80). Voilรก! Here we go. IPv6 traffic is now routed to IPv4. I repeated it with port 443 and the setup is done. And LetsEncrypt servers are happy too ๐Ÿ™‚

Categories
Apache Linux Perl

How to automate LetsEncrypt

A new service is born: Let’s Encrypt. It offers free SSL certificates that you can use for web servers, email servers or whatever service you want to secure with TLS. This blog post presents my strategy to automate certificate creation and renewal. Please, install Let’s Encrypt on your web server box before you start to follow the presented strategy.

The key to success is to have Let’s Encrypt running without any further interaction. I use webroot authentication – which allows me to leave the productive web service up and running while the certificates are being issued or renewed. Therefore, I created a file named “myserver.ini” in folder /etc/letsencrypt. This configuration file contains all details that are required for the certification process;

1
2
3
4
5
6
7
8
rsa-key-size = 4096
authenticator = webroot
webroot-path = /path/to/webroot/
server = https://acme-v01.api.letsencrypt.org/directory
renew-by-default = True
agree-tos
email = <my-email-address>
domains = domain1.com, domain2.com

The second component of my strategy is the central piece: a script called “renewCertificates.pl”:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
#!/usr/bin/perl
 
my $DOMAINS = {
    'myserver' => {
        'configFile' => '/etc/letsencrypt/myserver.ini',
        'leSubDir'   => 'domain1.com',
        'certDir'    => '/var/www/domain1.com/certs',
    },
};
 
my $domain;
my $renewed = 0;
chdir ('/usr/local/letsencrypt');
foreach $domain (keys(%{$DOMAINS})) {
    print "INFO  - $domain - START\n";
    my $cmd = '/usr/local/scripts/checkCertExpiry.sh 30 '.$DOMAINS->{$domain}->{'certDir'}.'/cert.pem >/dev/null';
    my $rc = system($cmd);
    if ($rc) {
        $cmd = './letsencrypt-auto certonly --config '.$DOMAINS->{$domain}->{'configFile'}.' --renew-by-default';
        $rc = system($cmd);
        if (!$rc) {
            $cmd = 'cp /etc/letsencrypt/live/'.$DOMAINS->{$domain}->{'leSubDir'}.'/* '.$DOMAINS->{$domain}->{'certDir'}.'/';
            $rc = system($cmd);
            if ($rc) {
                print "ERROR - $domain - Cannot deploy\n";
            } else {
                print "INFO  - $domain - Deployed\n";
                $renewed = 1;
            }
        } else {
            print "ERROR - $domain - Cannot generate certificates\n";
        }
    } else {
        print "INFO  - $domain - Certificate does not expire within 30 days\n";
    }
    print "INFO  - $domain - END");
}
 
if ($renewed) {
   system("/etc/init.d/apache2 reload");
}
 
exit 0;

This scripts allows renewal of multiple certificates by supporting multiple configurations. Lines 3-9 describe these configurations. leSubDir (line 6) is the sub directory that Let’s Encrypt creates in the certification process. It is the name of the first domain specified in the configuration file, here: domain1.com. certDir (line 7) is the target path where the certificates will be deployed to.

A second script supports this procedure by telling whether a certificate will expire within a certain number of days (see line 16 above):

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
#!/bin/bash
 
# First parameter specifies if certificate expire in the next X days
DAYS=$1
 
target=$2
if [ ! -f "$target" ]; then
    echo "Certificate does not exist (RC=2)"
    exit 2;
fi
 
openssl x509 -checkend $(( 86400 * $DAYS )) -enddate -in "$target" >/dev/null 2>&1
expiry=$?
if [ $expiry -eq 0 ]; then
    echo "Certificate will not expire (RC=0)"
    exit 0
else
    echo "Certificate will expire (RC=1)"
    exit 1
fi

This script returns 0 when the given certificate will not expire, otherwise it returns a non-0 value. The Perl script above will renew certificates 30 days before expiration only.

The last piece is the Apache configuration to be used on these domains:

1
2
3
4
    SSLEngine on
    SSLCertificateFile /var/www/domain1.com/certs/cert.pem
    SSLCertificateKeyFile /var/www/domain1.com/certs/privkey.pem
    SSLCertificateChainFile /var/www/domain1.com/certs/fullchain.pem

I run the central Perl script above daily and do not need to worry about certificates anymore ๐Ÿ™‚

Categories
Bugzilla for Java

Bugzilla 4 Java V2.0.3 released

A new version 2.0.3 of B4J has been released. It ensures compatibility with latest Bugzilla releases and introduces a workaround for “Untrusted Authentication Request” errors on some installations. This is the complete list of changes:

Bug

  • [BFJ-86] – JiraRpcSession does not allow inheritance
  • [BFJ-87] – BugzillaHttpSession does not allow inheritance
  • [BFJ-88] – BugzillaRpcSession does not allow inheritance
  • [BFJ-89] – AbstractLazyRetriever uses Long.getLong() instead of LangUtils.getLong()
  • [BFJ-90] – Untrusted Authentication Request

Improvement

  • [BFJ-85] – Overcome Jira's search result limitation

New Feature

  • [BFJ-91] – Add login token usage for RPC usage

You can download the new version here or visit the Homepage of the utility where you will find more documentation.

The Maven Site is available as well. The Maven coordinates are:

<dependency>
      <groupId>eu.ralph-schuster</groupId>
      <artifactId>b4j</artifactId>
      <version>2.0.3</version>
</dependency>