Sunday, August 28, 2016

Linux ACK counter limit security flaw

Researchers have discovered a flaw in the Linux kernel’s TCP/IP networking subsystem implementation of the RFC 5961 challenge ACK rate limiting, that could allow an off-path attacker to inject payload into unsecured TCP connections.

The 3.6 Linux kernel introduced a global challenge ACK counter limit in order to improve tcp's robustness to blind in-window attacks as specified in RFC 5961. However, an attacker can use this global challenge ACK counter to infer the sequence and ack number of an off-path tcp connection. In a typical client/server tcp connection, an attacker can establish connections with the server. Thus, the attacker can establish a number of connections with the server, and send sufficient out-of-window traffic, in order to use up the the entire global challenge ack limit. In this case, the attacker can expect to receive the number of challenge acks that is equal to the challenge ACK counter limit in response.    The attacker can then infer information about the sequence number and ack number of the connection by realizing if it has received fewer challenge ACKs in response than the global challenge ACK counter limit.

An example of the attack which makes use of this can be seen in the below video:

The most fastest way to resolve the issue to ensure you make a change to sysctl.conf by ensuring the below line is in your config:

net.ipv4.tcp_challenge_ack_limit = 999999999

This should ensure that the vulnerability is being resolved for now. 

Wednesday, June 22, 2016

Repair YUM on Oracle Linux

Whenever playing with Oracle Linux and trying out things on test systems you will break stuff at one point in time. As long as you except the fact that stuff might break when you try out new things this is not an issue. As long as you understand that production systems are not play systems there is also no issue. I recently hit an issue when I was trying to find ways to do an upgrade of Python. Even though all looked fine in the beginning I found out that it actually killed the way how some python scripts react. And more specifically how yum (a python script) reacted.

When trying to install some additional packages for another project by using yum the error message I received stated the below:

There was a problem importing one of the Python modules
required to run yum. The error leading to this problem was:

   No module named yum

Please install a package which provides this module, or
verify that the module is installed correctly.

It's possible that the above module doesn't match the
current version of Python, which is:
2.7.6 (default, Jun 14 2016, 09:18:35)
[GCC 4.4.7 20120313 (Red Hat 4.4.7-17)]

If you cannot solve this problem yourself, please go to
the yum faq at:

As it turns out the, yum is not able to work with the newer version of python that has become my standard python version (which I needed for Tensorflow). The issue can be resolved by calling the previous version explicitly.

If you check the content of /usr/bin/yum this will be as shown below:

import sys
    import yum
except ImportError:
    print >> sys.stderr, """\
There was a problem importing one of the Python modules
required to run yum. The error leading to this problem was:


Please install a package which provides this module, or
verify that the module is installed correctly.

It's possible that the above module doesn't match the
current version of Python, which is:

If you cannot solve this problem yourself, please go to
the yum faq at:

""" % (sys.exc_value, sys.version)

sys.path.insert(0, '/usr/share/yum-cli')
    import yummain
    yummain.user_main(sys.argv[1:], exit_code=True)
except KeyboardInterrupt, e:
    print >> sys.stderr, "\n\nExiting on user cancel."

As can be seen, the default version is called, by changing this to the previous version (do note my other blogpost on Python) you can repair yum again. After changing the first line to the one below yum worked again without any issue.


Tuesday, June 14, 2016

Upgrade Python on Oracle Linux

When deploying (currently) an Oracle Linux instance on the Oracle public compute cloud you will most likely get python version 2.6.6. The deployment version of Oracle Linux will provide you with Oracle Linux 6.6 configured in the manner as Oracle has prepared it for cloud deployment. Which is not that different from what you might install yourself.

The below screenshot from the Oracle compute cloud shows the version we will be using in this example.

As stated, this version will be shipping Python 2.6.6. In some cases you do want to upgrade your python version. We will upgrade Python in this example from Python 2.6.6 to a Python 2.7.6 version. Python is also shipping in a Python 3.x.x version, however, some changes to Python have been made which might render existing python code written under Python 2.x.x. unusable. For this reason we will stick with Python 2.7.6 while also perserving the Python 2.6.6 version on the system.

In esscence this is not making it a upgrade which will replace the old version, it is rather a Python 2.7.6 installation where we make Python 2.7.6 the default version instead of the Python 2.6.6 version.

Preparing the system
We will be compiling Python so we have to ensure we have the right development and build tooling installed. This can be done with yum by doing a group install and a install of a number of other packages as shown below.

yum groupinstall "Development tools"

yum install zlib-devel bzip2-devel openssl-devel ncurses-devel sqlite-devel readline-devel tk-devel gdbm-devel db4-devel libpcap-devel xz-devel

This ensures your Linux environment will have all the needed packages to do the compilation of Python 2.7.6

Downloading and compiling
We will be downloading the source code and compiling it into a workable version. A couple of things to keep in mind during the compilation are that we "need" to extent the configure command with the below to ensure the path in compiled into the executable during compilation.  LDFLAGS="-Wl,-rpath /usr/local/lib"

The following steps are needed to download, configure and build Python 2.7.6 into you systems:

cd /tmp
tar xf Python-2.7.6.tar.xz
cd Python-2.7.6
./configure --prefix=/usr/local --enable-unicode=ucs4 --enable-shared LDFLAGS="-Wl,-rpath /usr/local/lib"
make altinstall

With this done you should now have Python 2.7.6 installed on your system

Making Python 2.7.6 default
As we have installed Python 2.7.6 installed next to Python 2.6.6 the default version is still 2.6.6. you can check it in the same fashion as shown below

[root@tensor-0 bin]# which python
[root@tensor-0 bin]# python --version
Python 2.6.6
[root@tensor-0 bin]# 

As you can see python is found in /usr/bin/ and is currently version 2.6.6 while we would like Python 2.7.6 to be the default version.

You can achieve this by doing the following:
  • Rename /usr/bin/python to /usr/bin/python to /usr/bin/python2.6
  • softlink /usr/local/bin/python2.7 to /usr/bin/python (as shown below)

ln -s /usr/local/bin/python2.7 /usr/bin/python

This should ensure that you now have Python version 2.7.6 as the prime version when calling it. You can again check this by doing a python --version command which now should show 2.7.6 instead of 2.6.6

Resolved - Linux - unprotected private key file

When connecting to other systems using SSH in a key based manner you might have situations sometimes where you move your privtae key from one machine to another. For example when you have a new workstation you want your keys on this machine as they match the keys on other systems and the combiantion is needed to make the connection.

When you simply place them on your new machine without any other actions and connect to a remote SSH deamon it can happen that you will see the below message. (might differ slightly).

Permissions 0744 for '/home/jlo/.ssh/id_rsa' are too open.
It is recommended that your private key files are NOT accessible by others.
This private key will be ignored.
bad permissions: ignore key: /home/jlo/.ssh/id_rsa

This is your local machine telling you that your key is not having the appropriate permissions. Or in other words, the permissions on the key are to open and not secure enought. To resolve this you will have to execute a chmod. In our example, a chmod on these two files:

chmod 600 ~/.ssh/id_rsa
chmod 600 ~/.ssh/

This should resolve the issue for you and you should be able to connect via SSH in a key based manner without any issues. 

Monday, June 13, 2016

How to Manage and Monitor the Oracle ZFS Storage Appliance

Oracle storage solutions in the form of ZFS appliances are widely used in oracle focused and none-oracle focused IT footprints. This is for good reason as Oracle provides a good storage solution with the ZFS storage appliance range. I already posted a number of blogposts on ZFS, including on how to analyse the ZFS storage appliance and on how to use the ZFS storage appliance simulator.

The Oracle ZFS storage appliance simulator is a Oracle VirtualBox appliance which can be used to use for testing and learning purposes.

To add to the already mentioned posts is now this post, providing you the below video on how to manage and monitor the Oracle ZFS storage appliance which might provide you additional information when exploring the options of ZFS appliances.

Monday, June 06, 2016

Oracle DBaaS Cloud Listener

When developing a private database cloud to provide Database as a Service (DBaaS) solutions to internal customers this will require a different way of looking at common concepts opposed to more static environments. In the attached paper the concept of the Cloud Listener is explained. The Cloud Listener concept is a concept based upon the Oracle Remote listener concept which is more commonly used in environments.

Using the concept of the Oracle Remote Listener to have user connecting to the database is an ideal concept to enrich the experience of a private cloud serving databases as a service to internal users.

This concept, as outlined in the paper, is a tested and proven solution which has been implemented by the authors at customers who use large numbers of databases in a highly agile environment. The environment where this has been used are commonly full Oracle based environments where the overall IT footprint consists out of private cloud deployments based upon Oracle Linux, Oracle VM and Oracle Enterprise manger (including all self service options available within OEM).

The above whitepaper is written by Peter Lengkeek and Johan Louwers.

Friday, June 03, 2016

Oracle with Docker and Openstack

Oracle is a supporter of both Docker and and Openstack. They provide a solution using Docker and Openstack in combination with Oracle Linux and Oracle Solaris. To understand more of the benefits of both Docker and Openstack please refer to the post I recently wrote on the website which will outline this in more detail.

This post will give you some insights into why the combination of both Docker and Openstack is a solution enterprises want to look into when developing there next IT footprint for the future. 

Thursday, May 19, 2016

Oracle Linux - Clone file permissions with chmod

Every now and then file permissions under Linux can be tricky. in some cases a wrong file permission can make it happen that things do not work they way you would expect them. Also I found that a lot of people find it challenging to set the correct file permissions using the command line under Linux. A way to make life more easy in some cases is to use the option to "clone" file permissions with a single command.

For example, if you have created some addons to a tool running on an Oracle Linux system and you want to the addon file to have the same permissions as another file you can use the --reference option from the chmod command.

As an example we have two .jar files:

[root@localhost ~]# ls -l *.jar
-rwxr-xr-x. 1 root root 88 May 19 10:48 addonExecution.jar
-rw-r--r--. 1 root root 11 May 19 10:48 executionLib.jar

We want to make sure that the addonExecution.jar has exactly the same permissions as the executionLib.jar file. We can do this by specifying the desired stated in a chmod command, we can also use the --reference option as shown below:

[root@localhost ~]# chmod --reference=executionLib.jar addonExecution.jar

This will make sure that the addonExecution.jar file now has exactly the same permissions as the file used as a reference.

[root@localhost ~]# ls -l *.jar
-rw-r--r--. 1 root root 88 May 19 10:48 addonExecution.jar
-rw-r--r--. 1 root root 11 May 19 10:48 executionLib.jar

Another use case example of this is that you can use it in a bash script where you might not be sure what the permissions should be for a certain file and only know that they always need to be the same as a specified other file. By using the --reference option you do not explicitly need to know the permissions during the creation of the bash script, you only need to know which file can be used as a reference. 

Oracle Linux - remove duplicate lines with awk

Sometimes you want to clean data quickly and remove all duplicate lines that are present in the file. For example a raw output from a system that is "dumped" on your Linux file system needs to be cleaned before you use it as input into another system. You can write some fancy code to do so, you can also use a very simple and straight forward solution by using awk on your Oracle Linux bash shell.

In the below example we have a file (the data with the duplicate lines) called rawdata.txt and we want to make a clean file called cleandata.txt. The example awk command can be used to read rawdata.txt and write the clean data to the file cleandata.txt

awk '!seen[$0]++' rawdata.txt >> cleandata.txt

The command itself is a very quick and dirty solution, most likely you want to use this in a wider script that is cleaning your data in a more sophisticated manner. 

Sunday, May 15, 2016

Oracle Linux Name Service Switch libraries

When scripting a bash solution which needs to check if a user is existing on your Oracle Linux instance you have a couple of options. The most known solution is to check if the username is present in the /etc/passwd file. You can simply do a cat of this file and using grep and wc command to make it more usable in your script. An example of this could be for example the command below which will give you the number of times that “apache” is mentioned in the file. Do remember, we assume this is the user apache and this is not very reliable in reality.

cat /etc/passwd | grep apache | wc –l

Another solution is making use of getent which is not that well known as the above example. The getent command displays entries from databases supported by the Name Service Switch libraries. An example of this is shown below:

[root@dev1 ~]# getent passwd apache
[root@dev1 ~]#

Where in case the user is not existing the command will provide no output:

[root@dev1 ~]# getent passwd apache222
[root@dev1 ~]#

Using a wc –l on getent will provide you a more pure answer opposed to a wc –l on a cat from the passwd file. As stated; The getent command displays entries from databases supported by the Name Service Switch libraries. To understand this in a bit more detail and understand what databases are that are supported by the Name Service Switch libraries you can check the configuration file. Under Oracle Linux (and most other Linux distributions) this can be found at /etc/nsswitch.conf .  An example of a standard nsswitch.conf file is shown below. As you can see a lot more is supported by the Name Service Switch libraries and not only passwd.

# /etc/nsswitch.conf
# An example Name Service Switch config file. This file should be
# sorted with the most-used services at the beginning.
# The entry '[NOTFOUND=return]' means that the search for an
# entry should stop if the search in the previous entry turned
# up nothing. Note that if the search failed due to some other reason
# (like no NIS server responding) then the search continues with the
# next entry.
# Valid entries include:
#       nisplus                 Use NIS+ (NIS version 3)
#       nis                     Use NIS (NIS version 2), also called YP
#       dns                     Use DNS (Domain Name Service)
#       files                   Use the local files
#       db                      Use the local database (.db) files
#       compat                  Use NIS on compat mode
#       hesiod                  Use Hesiod for user lookups
#       [NOTFOUND=return]       Stop searching if not found so far

# To use db, put the "db" in front of "files" for entries you want to be
# looked up first in the databases
# Example:
#passwd:    db files nisplus nis
#shadow:    db files nisplus nis
#group:     db files nisplus nis

passwd:     files
shadow:     files
group:      files

#hosts:     db files nisplus nis dns
hosts:      files dns

# Example - obey only what nisplus tells us...
#services:   nisplus [NOTFOUND=return] files
#networks:   nisplus [NOTFOUND=return] files
#protocols:  nisplus [NOTFOUND=return] files
#rpc:        nisplus [NOTFOUND=return] files
#ethers:     nisplus [NOTFOUND=return] files
#netmasks:   nisplus [NOTFOUND=return] files

bootparams: nisplus [NOTFOUND=return] files

ethers:     files
netmasks:   files
networks:   files
protocols:  files
rpc:        files
services:   files

netgroup:   nisplus

publickey:  nisplus

automount:  files nisplus
aliases:    files nisplus

Wednesday, May 11, 2016

Oracle Linux addons channel

When working with Oracle Linux and using the Oracle Linux YUM repository you might be missing out. It is worth while checking which channels are being activated for YUM as not all channels are active by default after an installation or might have been disabled by a template installation.

Something I do see commonly is that administrators who create Oracle Linux templates do disable a lot of the YUM channels. If you receive a template based installation in a private cloud setup it is good practice to first check which channels are activated and which have been disabled. If some of the channels, for example the addons channel, are disabled you might be missing out on a lot of things and find yourself in a “dependency hell”

Rather than trying to resolve this manually it is good to check the channel settings in /etc/yum/repos.d/* normally this will be /etc/yum.repos.d/public-yum-ol6.repo for OL6 machines.

A default installation will state enabled=0 for the public_ol6_adddons channel. To enable this you will have to state enabled=1

Even though it might sounds like a minor change, it is commonly overlooked in first instance. Additional channels are available for Oracle Linux from the public Oracle YUM server which might be of interest to administrators to enable.

Tuesday, May 10, 2016

Oracle Linux pre-install RPM for EBS

When deploying Oracle software on a Oracle Linux (or any other Linux distribution) you will have to set a number of pre-requisites. In some cases you need to set certain kernel parameters and ensure you have specific packages installed on your machine.

For the Oracle database there is already for a long time a way of doing this by installing a specific RPM. As this is providing a great way of preparing your environment for the software installation Oracle is also providing a similar mechanism for preparing your environment for the installation of Oracle e-Business suite.

You can install a pre-install RPM for Oracle e-Business Suite 12.1 and 12.2 which you can fetch from the addons channel at ULN. A best practice for creating a new Oracle Linux environment to run Oracle e-Business Suite is to do a minimal installation of Oracle Linux and install the pre-install RPM on top of this. Reason for this is that you will not have unneeded packages and functionality which might cause hindrance at a later stage.

  • In essence the pre-install RPM will undertake the following tasks:
  • Downloading and installing all software package versions and dependencies required for installing E-Business Suite R12 (12.1, 12.2)
  • Creating the users oracle and applmgr for use as owners of the database and application tiers respectively, while setting hard and soft shell resource limits
  • Updating kernel parameters in /etc/sysctl.conf to recommended values
  • Sets DNS resolver parameters in /etc/resolv.conf to minimum recommended values
  • Sets 'numa=off' in the kernel in the kernel commandline
  • Disables 'Transparent Huge Pages (THP)' if enabled 

For more information on the pre-install RPM’s please refer to the following notes on My Oracle Support: 761566.1 & 1330701.1

Tuesday, April 26, 2016

Oracle Hybrid Cloud

Recently I presented together with Marcel Giacomini from Oracle on Oracle public, private and hybrid cloud. The hybrid cloud is a direction I personally feel the market will move towards very quickly. Even though cloud companies would like to see enterprises adopting a full cloud model I think a majority of the large enterprises and companies will take the route of hybrid cloud first.

To see more on the capabilities around hybrid cloud from Oracle have a look at the deck we presented during Advantage You.

Wednesday, April 20, 2016

Oracle Linux Unsupported Packages

When running Oracle Linux you do not have to purchase a support contract from Oracle. You are perfectly fine running Oracle Linux without purchasing the support. However, in general, when running Oracle Linux in a business environment you would like to have the option to call in support when needed. This means that most companies do purchase the support and use it whenever needed. A general misunderstanding is that everything shipped by Oracle is also supported by Oracle.

In fact some (a limited) parts are not supported by Oracle while at the same time you will be able to find them in the Oracle Linux distribution and you have the option to install them and use them. The general misconception comes from the fact that most people understand that when you download and install additional software that is not provided by Oracle you will not get support. At the same time they expect everything shipped by Oracle to be under the support contract.

In case you are in doubt if a specific part is under support you might want to check the “Unsupported Packages from ISO” at the Oracle Linux website. This list (current date – Do check the latest version) has  the following packages:

  1.  ccs
  2.  cluster-cim
  3.  cluster-glue-libs-devel
  4.  clusterlib-devel
  5.  cluster-snmp
  6.  cman
  7.  cmirror
  8.  cmirror-standalone
  9.  corosynclib-devel
  10.  ctdb
  11.  ctdb-devel
  12.  dlm
  13.  fence-agents-all
  14.  fence-virtd-checkpoint
  15.  foghorn
  16.  gfs2-utils
  17.  haproxy
  18.  ipvsadm
  19.  keepalived
  20.  libesmtp-devel
  21.  luci
  22.  lvm2-cluster
  23.  lvm2-cluster-standalone
  24.  N/A
  25.  omping
  26.  openaislib-devel
  27.  pacemaker
  28.  pacemaker-doc
  29.  pacemaker-libs-devel
  30.  pcs
  31.  piranha
  32.  python-repoze-what-quickstart
  33.  resource-agents
  34.  rgmanager
  35.  ricci
  36.  xfsdump
  37.  xfsprogs
  38.  xfsprogs-devel

Monday, March 28, 2016

virtualbox only showing 32 bits options

I received  my new laptop from my work this week. In general I tend to be not that happy with receive a new laptop from work because it takes some time from your day to get everything back working again. However, this time I was more disappointed than normal as it turned out that I was not able to run 64 bit guests on my laptop and only 32 bits options where available.

After some checking I found out that my OS was a 64 bit OS and everything should work as far as I could see. However, only 32 bits options where available. As it turns out Windows 7 in combination with virtualbox is not allowing you to run 64 bit guests when certain virtualization is not enabled in the bios of your machine.

After turning on "Intel Virtualization Technology" and "Intel VT-d feature" virtualbox again allowed for running 64 bit guests.

Just a small reminder for everyone who runs into this issue. In case you have this enabled and it is not working, make sure that you disable Hyper-V on windows.

For more background information on this you can refer to ticket 12350 on the virtualbox website.