Wednesday, September 1, 2021

Jenkins Notes

Just a place for links to getting Jenkins working for CICD for my projects.

Installing

That gets you started. 

Plugins

Probably overkill and more than you need but I tried to cover most of the stuff I have in github. You may want to add others as well.
ace-editor
ant
antisamy-markup-formatter
apache-httpcomponents-client-4-api
authorize-project
bootstrap4-api
bootstrap5-api
bouncycastle-api
branch-api
build-timeout
caffeine-api
checks-api
cloudbees-credentials
cloudbees-folder
command-launcher
credentials
credentials-binding
deploy
display-url-api
durable-task
ec2-deployment-dashboard
echarts-api
email-ext
font-awesome-api
git
git-client
github
github-api
github-branch-source
git-server
gradle
handlebars
jackson2-api
jdk-tool
jjwt-api
jquery3-api
jsch
junit
ldap
lockable-resources
mailer
matrix-auth
matrix-project
momentjs
okhttp-api
pam-auth
pipeline-build-step
pipeline-github-lib
pipeline-graph-analysis
pipeline-input-step
pipeline-milestone-step
pipeline-model-api
pipeline-model-definition
pipeline-model-extensions
pipeline-rest-api
pipeline-stage-step
pipeline-stage-tags-metadata
pipeline-stage-view
plain-credentials
plugin-util-api
popper2-api
popper-api
resource-disposer
scm-api
script-security
snakeyaml-api
ssh-credentials
sshd
ssh-slaves
structs
timestamper
token-macro
trilead-api
windows-slaves
workflow-aggregator
workflow-api
workflow-basic-steps
workflow-cps
workflow-cps-global-lib
workflow-durable-task-step
workflow-job
workflow-multibranch
workflow-scm-step
workflow-step-api
workflow-support
ws-cleanup

Users


Build a Java app with Maven

You can jump past the Docker setup if you already did the above by scrolling down to Fork and clone the sample repository. Also ignore docker stuff in project setup.


Sorting Issues

I can't seem to login to Jenkins after restart.

Lots of posts say to do something like

sed -i 's/<useSecurity>true<\/useSecurity>/<useSecurity>false<\/useSecurity>/g' /var/lib/jenkins/config.xml

but that assumes you set up with a local DB instead of the linux users. If you make the change and are using the local auth Jenkins will not even start. Worse yet, it fails quietly wasting even more time digging uselessly to try and find an error to solve. Here is the rub. In order to use the local auth system the user Jenkins runs at needs to be able to read /etc/shadow which means running root or hacking perms. So despite using the local users making the most sense logistically, it makes little sense security wise so you want to switch to something else but first you need to get in. The way that works is to set the password for the user you are running Jenkins as

Now you can login with that user and password and change the Security Realm in Configure Global Security to something else. Jenkins’ own user database makes the most sense for me.

Jenkins Configure Global Security page

Info on options can be found here.

To be closer to my work setup I made it a bit fancier.

Project-based Matrix Authorization Strategy example

If you do the above project based auth you will also want to config Access Control for Builds

To make life somewhat simpler though I'm defaulting to running builds as the jenkins user. To sort perms then I needed to run
chown jenkins.jenkins -R /var/lib/jenkins/workspace/

You might also want to look at How to Install Role Strategy Plugin in Jenkins

Could not lock config file../.git/config :permission denied in Jenkins build

I hit this because I added a project and built it before restarting so everything was owned by and restricted to root.


Thursday, July 22, 2021

A2Hosting unmanaged VPS set up

What did not work

Configure a Dedicated Server with Webmin

External email just did not work. Tried lots of fixes but nothing seemed to sort it.

Install iRedMail on Red Hat Enterprise Linux, CentOS

Seemed to collide with Apache so web stuff did not work and neither did any email. started to try and repair but basically started looking like starting over so I did.

Got most of the way through. Was installing tons of extra stuff and started hitting stuff that was no longer supported so started over yet again.

What did

How to Install an Operating System on a Virtualizor Unmanaged VPS

Unmanaged VPS Quick Start Guide

setup ssh keys

yum install yum-utils

yum install dos2unix

How to Secure Your Server

Downloading and Installing Virtualmin

Canceled out when installer wanted to start messing with network interfaces but clear email local and external with auth works.

Created redirect to old server in index.html to hide website under construction.

vi /var/www/html/index.html

Change it to

<html>
<head>
<meta http-equiv=refresh content=0;url=http://dea42.com>
</head>
</html>


vi /var/www/html/info.php
add to the file
<?php phpinfo(); ?>
goto http://your_server_IP_address/info.php to confirm is working and it was not. 

Given compatibility issues above PHP 7.2 sticking with 7.2 for now.

yum install http://rpms.remirepo.net/enterprise/remi-release-7.rpm

yum-config-manager --enable remi-php72

yum -y install mod_ssl php php-mysql php-mbstring phpmyadmin php-intl

echo "RequestHeader unset Proxy early" >> /etc/httpd/conf/httpd.conf

to access phpMyAdmin from another machine you will need to edit

/etc/httpd/conf.d/phpMyAdmin.conf In my case I just added the ip I was working from 70.121.71.135 but you could comment out / remove the RequireAny sections to allow any IP.

<Directory /usr/share/phpMyAdmin/>

   AddDefaultCharset UTF-8

   <IfModule mod_authz_core.c>

     # Apache 2.4

     <RequireAny>

       Require ip 127.0.0.1

       Require ip 70.121.71.135

       Require ip ::1

     </RequireAny>

   </IfModule>

   <IfModule !mod_authz_core.c>

     # Apache 2.2

     Order Deny,Allow

     Deny from All

     Allow from 127.0.0.1

     Allow from ::1

   </IfModule>

</Directory>

<Directory /usr/share/phpMyAdmin/setup/>

   <IfModule mod_authz_core.c>

     # Apache 2.4

     <RequireAny>

       Require ip 127.0.0.1

       Require ip 70.121.71.135

       Require ip ::1

     </RequireAny>

   </IfModule>

   <IfModule !mod_authz_core.c>

     # Apache 2.2

     Order Deny,Allow

     Deny from All

     Allow from 127.0.0.1

     Allow from ::1

   </IfModule>

</Directory>

Be sure date.timezone is set in /etc/php.ini

service httpd restart

Check PHP again and if working
rm -f  /var/www/html/info.php

Remote DB logins are still disabled so phpMyAdmin will still not let you login yet. You can't even run 
mysql -u root -p
so run
mysql_secure_installation
mysql -u root -p
CREATE USER 'notRoot'@'%' IDENTIFIED  BY 'P@$$w0rd';
grant all privileges on *.* to 'notRoot'@'%';
flush privileges;

Better to set default time zone in /etc/my.cnf under [mysqld]
default-time-zone=America/Chicago

After running installer I found I needed to add
$config['enigma_pgp_homedir'] = 'plugins/enigma';
to the end of config/config.inc.php

And finish SSL setup by changing conf.d/ssl.conf to
Listen 443 https
SSLPassPhraseDialog exec:/usr/libexec/httpd-ssl-pass-dialog
SSLSessionCache         shmcb:/run/httpd/sslcache(512000)
SSLSessionCacheTimeout  300
SSLRandomSeed startup file:/dev/urandom  256
SSLRandomSeed connect builtin
SSLCryptoDevice builtin
<VirtualHost _default_:443>
ErrorLog logs/ssl_error_log
TransferLog logs/ssl_access_log
LogLevel warn
SSLEngine on
SSLProtocol all -SSLv2 -SSLv3
SSLCipherSuite HIGH:3DES:!aNULL:!MD5:!SEED:!IDEA
SSLCertificateFile /etc/webmin/letsencrypt-cert.pem
SSLCertificateKeyFile /etc/webmin/letsencrypt-key.pem
SSLCertificateChainFile /etc/webmin/letsencrypt-ca.pem
<Files ~ "\.(cgi|shtml|phtml|php3?)$">
    SSLOptions +StdEnvVars
</Files>
<Directory "/var/www/cgi-bin">
    SSLOptions +StdEnvVars
</Directory>
BrowserMatch "MSIE [2-5]" \
         nokeepalive ssl-unclean-shutdown \
         downgrade-1.0 force-response-1.0
CustomLog logs/ssl_request_log \
          "%t %h %{SSL_PROTOCOL}x %{SSL_CIPHER}x \"%r\" %b"
</VirtualHost>

Now you need to edit or create .htaccess file in your domain root directory and add these lines to redirect http to https.

Options +SymLinksIfOwnerMatch
RewriteEngine On
RewriteCond %{HTTPS}  !=on
RewriteRule ^(.*)$ https://%{HTTP_HOST}%{REQUEST_URI} [L,R=301,NE]
Header always set Content-Security-Policy "upgrade-insecure-requests;"

Before using a .htaccess file, we need to update the AllowOverride setting to be able to overwrite Apache directives.

vi /etc/httpd/conf/httpd.conf

 

Locate the <Directory /var/www/html> section and change the AllowOverride directive from None to All:

/etc/httpd/conf/httpd.conf
. . .
<Directory /var/www/html>
. . .
 #
 # AllowOverride controls what directives may be placed in .htaccess files.
 # It can be "All", "None", or any combination of the keywords:
 # Options FileInfo AuthConfig Limit
 #
 AllowOverride All
. . .
</Directory>
. . .
 

Save and exit the file and then restart Apache to apply the change:

service httpd restart


At this point I tried several things attempting to get email working but still did not have incoming working when I got distracted by all the failed login attempts getting logged but not banned so I stopped to config fail2ban. 


When I came back to email it was working. So here are the active lines in the postfix config file to get domain (dea42.org) with catchall set to deliver to local user deabigt.

main.cf

command_directory = /usr/sbin
daemon_directory = /usr/libexec/postfix
data_directory = /var/lib/postfix
mydomain = dea42.org
inet_protocols = all
mydestination = $myhostname, localhost.$mydomain, localhost, server.dea42.org
unknown_local_recipient_reject_code = 550
alias_maps = hash:/etc/aliases
alias_database = hash:/etc/aliases
debug_peer_level = 2
debugger_command =
         PATH=/bin:/usr/bin:/usr/local/bin:/usr/X11R6/bin
         ddd $daemon_directory/$process_name $process_id & sleep 5
sendmail_path = /usr/sbin/sendmail.postfix
newaliases_path = /usr/bin/newaliases.postfix
mailq_path = /usr/bin/mailq.postfix
setgid_group = postdrop
html_directory = no
manpage_directory = /usr/share/man
sample_directory = /usr/share/doc/postfix-2.10.1/samples
readme_directory = /usr/share/doc/postfix-2.10.1/README_FILES
virtual_alias_maps = hash:/etc/postfix/virtual
sender_bcc_maps = hash:/etc/postfix/bcc
sender_dependent_default_transport_maps = hash:/etc/postfix/dependent
mailbox_command = /usr/bin/procmail-wrapper -o -a $DOMAIN -d $LOGNAME
home_mailbox = Maildir/
smtpd_sasl_auth_enable = yes
smtpd_tls_security_level = may
broken_sasl_auth_clients = yes
smtpd_recipient_restrictions = permit_mynetworks permit_sasl_authenticated reject_unauth_destination
smtp_tls_security_level = may
mailbox_size_limit = 0
allow_percent_hack = no
smtpd_tls_key_file = /etc/webmin/letsencrypt-key.pem
smtpd_tls_CAfile = /etc/webmin/letsencrypt-ca.pem
smtpd_tls_cert_file = /etc/webmin/letsencrypt-cert.pem
smtpd_relay_restrictions = permit_mynetworks permit_sasl_authenticated defer_unauth_destination
myorigin = $mydomain
myhostname = mail.dea42.org
virtual_alias_domains = dea42.org

master.cf

smtp    inet    n       -       n       -       -       smtpd -o smtpd_sasl_auth_enable=yes -o smtpd_tls_security_level=may
pickup    unix  n       -       n       60      1       pickup
cleanup   unix  n       -       n       -       0       cleanup
qmgr      unix  n       -       n       300     1       qmgr
tlsmgr    unix  -       -       n       1000?   1       tlsmgr
rewrite   unix  -       -       n       -       -       trivial-rewrite
bounce    unix  -       -       n       -       0       bounce
defer     unix  -       -       n       -       0       bounce
trace     unix  -       -       n       -       0       bounce
verify    unix  -       -       n       -       1       verify
flush     unix  n       -       n       1000?   0       flush
proxymap  unix  -       -       n       -       -       proxymap
proxywrite unix -       -       n       -       1       proxymap
smtp      unix  -       -       n       -       -       smtp
relay     unix  -       -       n       -       -       smtp
showq     unix  n       -       n       -       -       showq
error     unix  -       -       n       -       -       error
retry     unix  -       -       n       -       -       error
discard   unix  -       -       n       -       -       discard
local     unix  -       n       n       -       -       local
virtual   unix  -       n       n       -       -       virtual
lmtp      unix  -       -       n       -       -       lmtp
anvil     unix  -       -       n       -       1       anvil
scache    unix  -       -       n       -       1       scache
submission      inet    n       -       n       -       -       smtpd -o smtpd_sasl_auth_enable=yes -o smtpd_tls_security_level=may
smtps   inet    n       -       n       -       -       smtpd -o smtpd_sasl_auth_enable=yes -o smtpd_tls_security_level=may -o smtpd_tls_wrappermode=yes

Not receiving email : Relay access denied in Postfix (CentOS 7) - fixing virtual hosts

For a catchall add @domain    user@localhost to /etc/postfix/virtual
@dea42.org      deabigt@localhost
Note @localhost is required to deliver to a local user.

Added second domain with small PHP site and catchall email.

To see who the email was sent to Change message list columns to add To column to list view.

Install Tomcat




ProFTP
Change the certs to use the Let's Encrypt ones in /etc/proftpd/conf.d/virtualmin.conf
TLSRSACertificateFile         /etc/webmin/letsencrypt-cert.pem
TLSRSACertificateKeyFile      /etc/webmin/letsencrypt-key.pem

TODO:

How to Enable Two-Factor Authentication in Webmin / Virtualmin / Usermin / Cloudmin

Add Procmail filters Also see


Notes

Spring Boot: Create Custom Jsp Tag - Unable to find taglib

MySQL quick ref

Monday, February 8, 2021

Comment on NY Times cell phone tracking piece

A friend posted this article about cell phone tracking (kind of the flip side to their earlier and much longer article on phone tracking) and I thought it needed more than a Facebook post to fill in some of the stuff missing from the article and expand on other bits. Not to increase paranoia or dismiss it but to point out things those not in IT might not know. Especially given the increasing number of posts I see about tracking worries that are way off the mark (tracking chips in vaccines for example) that often would be less of an issue even if they were real than what is actually in increasing use. I don't have an answer. Some think all tracking info should not be kept. On the other end are those that are not worried at all. In the end everyone has their own idea of acceptable risk but that should be based on actual risk factors and as the NY Times' article points out companies are making a business out of linking all available data sources.

Location tracking basics

Here is the rub. Every tool can be used as a weapon. Cell phone location ping data is just a tiny fraction of the stream:

  • Apps track your location as well as what you did. Many apps ask for WAY more perms than they need. Some because the developer was lazy, some to have data to monetize.
  • WiFi access points can track you as you pass them even when you do not connect to them. This includes hotspots in your car. For example 

  • Linked security camera and WAMI systems can basically TiVo an area to allow forward and backward tracking of multiple objects over the entire coverage area which is growing fast. You might be amazed how many posts I see of people having or wanting license plate readers to log cars passing their house. I might note too want a Pan, Tilt  and Zoom (PTZ) camera because they think it will somehow locate and track people in range. More here to why they won't. So if WAMI for homes became available there would probably many that would sign up.
  • Bluetooth is getting strong enough these days to be used as well and is.
  • Then there is LoRa which Amazon just adopted as part of Sidewalk and so on. 
  • Not to mention almost everything is or will shortly be connect to the internet. If it can ID you (as in you have any sort of auth on it) there is another data point. 

That means it is virtually impossible to not be tracked now and it will only get harder as data storage rates drop, computing power increases and systems get deployed and continue to interconnect. The thing is that the data is huge so without a reason it generally just gets stored for "in case". Developers will always err on the side of having too much data to debug with and for future features. Companies are always looking for new features to offer customers and ways to monetize having to store the data they have. And all too often security and QA are seen as overhead costs to be kept to a min. Add to that governments will always want more and better tools the track down "criminals" and most people will be happy to comply as long as they assume it will never be used against them or they think they can exclude themselves. Look at all the people commenting on rep posts without constituent badges because they seem to think that means Facebook will not know where they are even though if you log in from a diff browser you get a notification telling you your account was just access from machine X in city Y. Failed logins are logged too. Even by basic hosted website has that level of tracking by default so you can imagine what a site like Facebook is doing. Even a non static IP is linked to a location for the length of the exchange. Granted there are ways to hide your location somewhat but few are going to deal with that hassle even if they are aware of them. Plus many proxies and VPNs are not as anomilizing as they claim. Also, one of the things all the breeches have shown is most people still use the same login and password everywhere and do not even know what 2FA is making them both easier to track and hack. There again it is not that hard to track down someone's home address these days from a few data points given a lot of government data is online and or sold to search engines like PublicData.com (since 1997).  Outrage back when people first heard and they moved the servers off shore but even then that only saved you the trip to the clerk for the data. People just did not know. While we are on social media you might want to look at this story about the police tracked some protesters via social media.


So then it would seem the only option is to limit access to the data or what the data can be used for. But I would not hold much hope there either. Take WAMI for example. Back in 2016 a company did a WAMI test that was pretty successful at tracking criminals. People kind of freaked at the time but Baltimore police are attempting a new test run to track how it affects crime when people know it is watchingPeople seem freaked about drones in general though the WAMI tests are with piloted aircraft people still seem to see them as drones. You know if they get their test and it shows good results others will want it too. Same goes for other related tech. As I mentioned above people are saying they want features like tracking and recognition even if saying they do not want the government and companies them. The Chinese are going all in so in the end it might be a question if we buy from them or develop it here. BTW did you know almost all security cameras are made in China? There are people warning about that as well of course. That is a whole other thread about unsecured networks of IoT devices whether or not they came with backdoors.