Wednesday, September 1, 2021

Jenkins Notes

Just a place for links to getting Jenkins working for CICD for my projects.

Installing

That gets you started. 

Plugins

Probably overkill and more than you need but I tried to cover most of the stuff I have in github. You may want to add others as well.
ace-editor
ant
antisamy-markup-formatter
apache-httpcomponents-client-4-api
authorize-project
bootstrap4-api
bootstrap5-api
bouncycastle-api
branch-api
build-timeout
caffeine-api
checks-api
cloudbees-credentials
cloudbees-folder
command-launcher
credentials
credentials-binding
deploy
display-url-api
durable-task
ec2-deployment-dashboard
echarts-api
email-ext
font-awesome-api
git
git-client
github
github-api
github-branch-source
git-server
gradle
handlebars
jackson2-api
jdk-tool
jjwt-api
jquery3-api
jsch
junit
ldap
lockable-resources
mailer
matrix-auth
matrix-project
momentjs
okhttp-api
pam-auth
pipeline-build-step
pipeline-github-lib
pipeline-graph-analysis
pipeline-input-step
pipeline-milestone-step
pipeline-model-api
pipeline-model-definition
pipeline-model-extensions
pipeline-rest-api
pipeline-stage-step
pipeline-stage-tags-metadata
pipeline-stage-view
plain-credentials
plugin-util-api
popper2-api
popper-api
resource-disposer
scm-api
script-security
snakeyaml-api
ssh-credentials
sshd
ssh-slaves
structs
timestamper
token-macro
trilead-api
windows-slaves
workflow-aggregator
workflow-api
workflow-basic-steps
workflow-cps
workflow-cps-global-lib
workflow-durable-task-step
workflow-job
workflow-multibranch
workflow-scm-step
workflow-step-api
workflow-support
ws-cleanup

Users


Build a Java app with Maven

You can jump past the Docker setup if you already did the above by scrolling down to Fork and clone the sample repository. Also ignore docker stuff in project setup.


Sorting Issues

I can't seem to login to Jenkins after restart.

Lots of posts say to do something like

sed -i 's/<useSecurity>true<\/useSecurity>/<useSecurity>false<\/useSecurity>/g' /var/lib/jenkins/config.xml

but that assumes you set up with a local DB instead of the linux users. If you make the change and are using the local auth Jenkins will not even start. Worse yet, it fails quietly wasting even more time digging uselessly to try and find an error to solve. Here is the rub. In order to use the local auth system the user Jenkins runs at needs to be able to read /etc/shadow which means running root or hacking perms. So despite using the local users making the most sense logistically, it makes little sense security wise so you want to switch to something else but first you need to get in. The way that works is to set the password for the user you are running Jenkins as

Now you can login with that user and password and change the Security Realm in Configure Global Security to something else. Jenkins’ own user database makes the most sense for me.

Jenkins Configure Global Security page

Info on options can be found here.

To be closer to my work setup I made it a bit fancier.

Project-based Matrix Authorization Strategy example

If you do the above project based auth you will also want to config Access Control for Builds

To make life somewhat simpler though I'm defaulting to running builds as the jenkins user. To sort perms then I needed to run
chown jenkins.jenkins -R /var/lib/jenkins/workspace/

You might also want to look at How to Install Role Strategy Plugin in Jenkins

Could not lock config file../.git/config :permission denied in Jenkins build

I hit this because I added a project and built it before restarting so everything was owned by and restricted to root.


Thursday, July 22, 2021

A2Hosting unmanaged VPS set up

What did not work

Configure a Dedicated Server with Webmin

External email just did not work. Tried lots of fixes but nothing seemed to sort it.

Install iRedMail on Red Hat Enterprise Linux, CentOS

Seemed to collide with Apache so web stuff did not work and neither did any email. started to try and repair but basically started looking like starting over so I did.

Got most of the way through. Was installing tons of extra stuff and started hitting stuff that was no longer supported so started over yet again.

What did

How to Install an Operating System on a Virtualizor Unmanaged VPS

Unmanaged VPS Quick Start Guide

setup ssh keys

yum install yum-utils

yum install dos2unix

How to Secure Your Server

Downloading and Installing Virtualmin

Canceled out when installer wanted to start messing with network interfaces but clear email local and external with auth works.

Created redirect to old server in index.html to hide website under construction.

vi /var/www/html/index.html

Change it to

<html>
<head>
<meta http-equiv=refresh content=0;url=http://dea42.com>
</head>
</html>


vi /var/www/html/info.php
add to the file
<?php phpinfo(); ?>
goto http://your_server_IP_address/info.php to confirm is working and it was not. 

Given compatibility issues above PHP 7.2 sticking with 7.2 for now.

yum install http://rpms.remirepo.net/enterprise/remi-release-7.rpm

yum-config-manager --enable remi-php72

yum -y install mod_ssl php php-mysql php-mbstring phpmyadmin php-intl

echo "RequestHeader unset Proxy early" >> /etc/httpd/conf/httpd.conf

to access phpMyAdmin from another machine you will need to edit

/etc/httpd/conf.d/phpMyAdmin.conf In my case I just added the ip I was working from 70.121.71.135 but you could comment out / remove the RequireAny sections to allow any IP.

<Directory /usr/share/phpMyAdmin/>

   AddDefaultCharset UTF-8

   <IfModule mod_authz_core.c>

     # Apache 2.4

     <RequireAny>

       Require ip 127.0.0.1

       Require ip 70.121.71.135

       Require ip ::1

     </RequireAny>

   </IfModule>

   <IfModule !mod_authz_core.c>

     # Apache 2.2

     Order Deny,Allow

     Deny from All

     Allow from 127.0.0.1

     Allow from ::1

   </IfModule>

</Directory>

<Directory /usr/share/phpMyAdmin/setup/>

   <IfModule mod_authz_core.c>

     # Apache 2.4

     <RequireAny>

       Require ip 127.0.0.1

       Require ip 70.121.71.135

       Require ip ::1

     </RequireAny>

   </IfModule>

   <IfModule !mod_authz_core.c>

     # Apache 2.2

     Order Deny,Allow

     Deny from All

     Allow from 127.0.0.1

     Allow from ::1

   </IfModule>

</Directory>

Be sure date.timezone is set in /etc/php.ini

service httpd restart

Check PHP again and if working
rm -f  /var/www/html/info.php

Remote DB logins are still disabled so phpMyAdmin will still not let you login yet. You can't even run 
mysql -u root -p
so run
mysql_secure_installation
mysql -u root -p
CREATE USER 'notRoot'@'%' IDENTIFIED  BY 'P@$$w0rd';
grant all privileges on *.* to 'notRoot'@'%';
flush privileges;

Better to set default time zone in /etc/my.cnf under [mysqld]
default-time-zone=America/Chicago

After running installer I found I needed to add
$config['enigma_pgp_homedir'] = 'plugins/enigma';
to the end of config/config.inc.php

And finish SSL setup by changing conf.d/ssl.conf to
Listen 443 https
SSLPassPhraseDialog exec:/usr/libexec/httpd-ssl-pass-dialog
SSLSessionCache         shmcb:/run/httpd/sslcache(512000)
SSLSessionCacheTimeout  300
SSLRandomSeed startup file:/dev/urandom  256
SSLRandomSeed connect builtin
SSLCryptoDevice builtin
<VirtualHost _default_:443>
ErrorLog logs/ssl_error_log
TransferLog logs/ssl_access_log
LogLevel warn
SSLEngine on
SSLProtocol all -SSLv2 -SSLv3
SSLCipherSuite HIGH:3DES:!aNULL:!MD5:!SEED:!IDEA
SSLCertificateFile /etc/webmin/letsencrypt-cert.pem
SSLCertificateKeyFile /etc/webmin/letsencrypt-key.pem
SSLCertificateChainFile /etc/webmin/letsencrypt-ca.pem
<Files ~ "\.(cgi|shtml|phtml|php3?)$">
    SSLOptions +StdEnvVars
</Files>
<Directory "/var/www/cgi-bin">
    SSLOptions +StdEnvVars
</Directory>
BrowserMatch "MSIE [2-5]" \
         nokeepalive ssl-unclean-shutdown \
         downgrade-1.0 force-response-1.0
CustomLog logs/ssl_request_log \
          "%t %h %{SSL_PROTOCOL}x %{SSL_CIPHER}x \"%r\" %b"
</VirtualHost>

Now you need to edit or create .htaccess file in your domain root directory and add these lines to redirect http to https.

Options +SymLinksIfOwnerMatch
RewriteEngine On
RewriteCond %{HTTPS}  !=on
RewriteRule ^(.*)$ https://%{HTTP_HOST}%{REQUEST_URI} [L,R=301,NE]
Header always set Content-Security-Policy "upgrade-insecure-requests;"

Before using a .htaccess file, we need to update the AllowOverride setting to be able to overwrite Apache directives.

vi /etc/httpd/conf/httpd.conf

 

Locate the <Directory /var/www/html> section and change the AllowOverride directive from None to All:

/etc/httpd/conf/httpd.conf
. . .
<Directory /var/www/html>
. . .
 #
 # AllowOverride controls what directives may be placed in .htaccess files.
 # It can be "All", "None", or any combination of the keywords:
 # Options FileInfo AuthConfig Limit
 #
 AllowOverride All
. . .
</Directory>
. . .
 

Save and exit the file and then restart Apache to apply the change:

service httpd restart


At this point I tried several things attempting to get email working but still did not have incoming working when I got distracted by all the failed login attempts getting logged but not banned so I stopped to config fail2ban. 


When I came back to email it was working. So here are the active lines in the postfix config file to get domain (dea42.org) with catchall set to deliver to local user deabigt.

main.cf

command_directory = /usr/sbin
daemon_directory = /usr/libexec/postfix
data_directory = /var/lib/postfix
mydomain = dea42.org
inet_protocols = all
mydestination = $myhostname, localhost.$mydomain, localhost, server.dea42.org
unknown_local_recipient_reject_code = 550
alias_maps = hash:/etc/aliases
alias_database = hash:/etc/aliases
debug_peer_level = 2
debugger_command =
         PATH=/bin:/usr/bin:/usr/local/bin:/usr/X11R6/bin
         ddd $daemon_directory/$process_name $process_id & sleep 5
sendmail_path = /usr/sbin/sendmail.postfix
newaliases_path = /usr/bin/newaliases.postfix
mailq_path = /usr/bin/mailq.postfix
setgid_group = postdrop
html_directory = no
manpage_directory = /usr/share/man
sample_directory = /usr/share/doc/postfix-2.10.1/samples
readme_directory = /usr/share/doc/postfix-2.10.1/README_FILES
virtual_alias_maps = hash:/etc/postfix/virtual
sender_bcc_maps = hash:/etc/postfix/bcc
sender_dependent_default_transport_maps = hash:/etc/postfix/dependent
mailbox_command = /usr/bin/procmail-wrapper -o -a $DOMAIN -d $LOGNAME
home_mailbox = Maildir/
smtpd_sasl_auth_enable = yes
smtpd_tls_security_level = may
broken_sasl_auth_clients = yes
smtpd_recipient_restrictions = permit_mynetworks permit_sasl_authenticated reject_unauth_destination
smtp_tls_security_level = may
mailbox_size_limit = 0
allow_percent_hack = no
smtpd_tls_key_file = /etc/webmin/letsencrypt-key.pem
smtpd_tls_CAfile = /etc/webmin/letsencrypt-ca.pem
smtpd_tls_cert_file = /etc/webmin/letsencrypt-cert.pem
smtpd_relay_restrictions = permit_mynetworks permit_sasl_authenticated defer_unauth_destination
myorigin = $mydomain
myhostname = mail.dea42.org
virtual_alias_domains = dea42.org

master.cf

smtp    inet    n       -       n       -       -       smtpd -o smtpd_sasl_auth_enable=yes -o smtpd_tls_security_level=may
pickup    unix  n       -       n       60      1       pickup
cleanup   unix  n       -       n       -       0       cleanup
qmgr      unix  n       -       n       300     1       qmgr
tlsmgr    unix  -       -       n       1000?   1       tlsmgr
rewrite   unix  -       -       n       -       -       trivial-rewrite
bounce    unix  -       -       n       -       0       bounce
defer     unix  -       -       n       -       0       bounce
trace     unix  -       -       n       -       0       bounce
verify    unix  -       -       n       -       1       verify
flush     unix  n       -       n       1000?   0       flush
proxymap  unix  -       -       n       -       -       proxymap
proxywrite unix -       -       n       -       1       proxymap
smtp      unix  -       -       n       -       -       smtp
relay     unix  -       -       n       -       -       smtp
showq     unix  n       -       n       -       -       showq
error     unix  -       -       n       -       -       error
retry     unix  -       -       n       -       -       error
discard   unix  -       -       n       -       -       discard
local     unix  -       n       n       -       -       local
virtual   unix  -       n       n       -       -       virtual
lmtp      unix  -       -       n       -       -       lmtp
anvil     unix  -       -       n       -       1       anvil
scache    unix  -       -       n       -       1       scache
submission      inet    n       -       n       -       -       smtpd -o smtpd_sasl_auth_enable=yes -o smtpd_tls_security_level=may
smtps   inet    n       -       n       -       -       smtpd -o smtpd_sasl_auth_enable=yes -o smtpd_tls_security_level=may -o smtpd_tls_wrappermode=yes

Not receiving email : Relay access denied in Postfix (CentOS 7) - fixing virtual hosts

For a catchall add @domain    user@localhost to /etc/postfix/virtual
@dea42.org      deabigt@localhost
Note @localhost is required to deliver to a local user.

Added second domain with small PHP site and catchall email.

To see who the email was sent to Change message list columns to add To column to list view.

Install Tomcat




ProFTP
Change the certs to use the Let's Encrypt ones in /etc/proftpd/conf.d/virtualmin.conf
TLSRSACertificateFile         /etc/webmin/letsencrypt-cert.pem
TLSRSACertificateKeyFile      /etc/webmin/letsencrypt-key.pem

TODO:

How to Enable Two-Factor Authentication in Webmin / Virtualmin / Usermin / Cloudmin

Add Procmail filters Also see


Notes

Spring Boot: Create Custom Jsp Tag - Unable to find taglib

MySQL quick ref