Firstyear's blog-a-log https://fy.blackhats.net.au Firstyear's blog Zola en Tue, 12 Mar 2024 00:00:00 +0000 SSH Key Authentication Basics Tue, 12 Mar 2024 00:00:00 +0000 Unknown https://fy.blackhats.net.au/blog/2024-03-12-ssh-key-auth/ https://fy.blackhats.net.au/blog/2024-03-12-ssh-key-auth/ <h1 id="ssh-key-authentication-basics">SSH Key Authentication Basics</h1> <p>SSH (Secure Shell) allows remotely accessing the command line interface (cli) of a remote machine. This is very useful for administration of a machine that may be in a completely different country or building.</p> <p>Because of this SSH is a very attractive target for attackers.</p> <h2 id="defaults">Defaults</h2> <p>The default is to login with a username and password.</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>┌───────┐ ┌───────┐ </span><span>│ │ │ │ </span><span>│ │───────Username─────────▶│ │ </span><span>│ │ │ │ </span><span>│ SSH │◀────Request Password────│ SSH │ </span><span>│Client │ │Server │ </span><span>│ │───────Password─────────▶│ │ </span><span>│ │ │ │ </span><span>│ │◀────────Success!────────│ │ </span><span>│ │ │ │ </span><span>└───────┘ └───────┘ </span></code></pre> <p>The problem with this model is that passwords can be bruteforced (weak passwords), they can be reused and then leaked in password breaches (credential stuffing). Generally SSH with only passwords is not a great idea.</p> <h2 id="asymmetric-key-cryptography">Asymmetric Key Cryptography</h2> <p>The basis of SSH keys lies in asymmetric key cryptography. This is where you have a private key and a public key.</p> <p>Given a piece of data, the private key can create a signature. The signature can be verified by the public key and the original data. However the public key can not create it's own signature.</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>┌─────────┐ ┌──────────┐ </span><span>│ │ │ │ </span><span>│ │ ┌─────────┐ │ │ </span><span>│ Private │ Signs │ │ │ Public │ </span><span>│ Key │───────┬─────▶ │Signature│ │ Key │ </span><span>│ │ │ │ │ │ │ </span><span>│ │ │ └─────────┘ │ │ </span><span>└─────────┘ │ └──────────┘ </span><span> │ </span><span> ┌────────┐ </span><span> │ │ </span><span> │ Data │ </span><span> │ │ </span><span> └────────┘ </span><span> </span><span> </span><span>┌─────────┐ ┌──────────┐ </span><span>│ │ │ │ </span><span>│ │ ┌─────────┐ │ │ </span><span>│ Private │ │ │ Verifies │ Public │ </span><span>│ Key │ │Signature│───────┬─────▶│ Key │ </span><span>│ │ │ │ │ │ │ </span><span>│ │ └─────────┘ │ │ │ </span><span>└─────────┘ │ └──────────┘ </span><span> │ </span><span> ┌────────┐ </span><span> │ │ </span><span> │ Data │ </span><span> │ │ </span><span> └────────┘ </span><span> </span><span> </span><span> </span><span>┌─────────┐ ┌──────────┐ </span><span>│ │ │ │ </span><span>│ │ ┌─────────┐ │ │ </span><span>│ Private │ │ │ Error! │ Public │ </span><span>│ Key │ │Signature│ X ◀───┬──────│ Key │ </span><span>│ │ │ │ │ │ │ </span><span>│ │ └─────────┘ │ │ │ </span><span>└─────────┘ │ └──────────┘ </span><span> │ </span><span> ┌────────┐ </span><span> │ │ </span><span> │ Data │ </span><span> │ │ </span><span> └────────┘ </span></code></pre> <p>These signatures are design to be mathematically infeasible to bruteforce or forge, meaning that they are very strong compared to passwords.</p> <h2 id="ssh-key-pairs">SSH Key Pairs</h2> <p>Using this knowledge we can now see how an SSH key works. The user (client) who wishes to authenticate to a server holds the private key, and the SSH server has the public key to verify the key is legitimate.</p> <p>In reverse, the server with the public key can't use that to authenticate since it is only able to verify signatures - not create them.</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>┌───────┐ ┌───────┐ </span><span>│ │ │ │ </span><span>│ │───────Username─────────▶│ │ </span><span>│ │ │ │ </span><span>│ │◀────────Challenge───────│ │ </span><span>│ │ │ │ </span><span>│ │────┐ │ │ </span><span>│ │ Private │ │ </span><span>│ │Key Signs │ │ </span><span>│ SSH │◀───┘ │ SSH │ </span><span>│Client │ │Server │ </span><span>│ │──────Signature─────────▶│ │ </span><span>│ │ │ │ </span><span>│ │ ┌─────│ │ </span><span>│ │ Public Key │ │ </span><span>│ │ Verifies │ │ </span><span>│ │ └────▶│ │ </span><span>│ │◀────────Success!────────│ │ </span><span>│ │ │ │ </span><span>└───────┘ └───────┘ </span></code></pre> <h2 id="practical-setup-guide-linux-macos">Practical Setup Guide (Linux, MacOS)</h2> <blockquote> <p>These commands should all be performed as your regular user account, unless otherwise stated.</p> </blockquote> <h3 id="creating-the-ssh-key-pair">Creating the SSH Key Pair</h3> <p>On your SSH client (your destkop, laptop etc). Create a new ssh key pair in the default location (<code>/home/$USER/.ssh/id_ecdsa</code>). You will be prompted for a passphrase. This is a passphrase to encrypt the private key when not in use.</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>ssh-keygen -t ecdsa </span></code></pre> <blockquote> <p>⚠️ <code>/home/$USER/.ssh/id_ecdsa</code> is the private key. Never reveal this to anyone!</p> </blockquote> <h3 id="add-the-public-key-to-the-server-ssh-copy-id">Add the Public Key to the Server (ssh-copy-id)</h3> <p>From the client you can automatically add the public key to the server. You will need to enter your username and password for this operation.</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>ssh-copy-id -f -i /home/$USER/.ssh/id_ecdsa.pub username@server-hostname </span></code></pre> <h3 id="add-the-public-key-to-the-server-manual">Add the Public Key to the Server (manual)</h3> <p>On the ssh client, view the public key. You may wish to copy-paste this files content.</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>cat /home/$USER/.ssh/id_ecdsa.pub </span></code></pre> <p>ssh to the server and run the following on the server.</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>cd ~ </span><span>mkdir .ssh </span><span>touch .ssh/authorized_keys </span><span>chmod 600 .ssh/authorized_keys </span><span>$EDITOR .ssh/authorized_keys </span></code></pre> <p>In the editor, paste your ssh public key (one per line) and then save and quit.</p> <h3 id="ssh-to-the-server">SSH to the server</h3> <p>Now type <code>ssh username@hostname</code> and you will use your key to authenticate!</p> <h2 id="hardening-the-server">Hardening the Server</h2> <p>Now that you have ssh key authentication configured you can harden your ssh server to only allow authentication with ssh keys.</p> <p>As root (<code>sudo -s</code>) add the following content to <code>/etc/ssh/sshd_config.d/hardening.conf</code></p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>UsePAM yes </span><span>PermitRootLogin no </span><span>PermitEmptyPasswords no </span><span>PasswordAuthentication no </span><span>AllowAgentForwarding no </span></code></pre> <p>Then restart sshd</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>systemctl restart sshd </span></code></pre> <p>Before logging out of root, ensure you can ssh to the server with key only. Try logging in with:</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>ssh -o PreferredAuthentications=password username@hostname </span></code></pre> <p>to ensure that password authentication is rejected correctly.</p> Webauthn Attestation and OpenSource Keys Sat, 02 Dec 2023 00:00:00 +0000 Unknown https://fy.blackhats.net.au/blog/2023-12-02-attestation-and-opensource/ https://fy.blackhats.net.au/blog/2023-12-02-attestation-and-opensource/ <h1 id="webauthn-attestation-and-opensource-keys">Webauthn Attestation and OpenSource Keys</h1> <p>Webauthn (Passkeys) are only going to become more important in the future and as this grows, deployments with higher security risks and criticality are going to need to start to understand and embrace attestation of their keys.</p> <p>In their current form, almost all software products and IDM's today allow you to enroll any cryptographic authenticator. It doesn't matter what make or model it is, it will be allowed.</p> <p>However, not all authenticators are made equal. They each have different properties, security features, and some even have security issues affecting their hardware or software. Because webauthn is a <em>self contained</em> multiple factor authenticator, this means we need to be even more careful to ensure these devices are secure.</p> <p>To manage this deployments can <em>attest</em> keys during enrollment. Attestation is a cryptographic proof of origin that proves the device is of a certain make and model. This allows us to select which devices we allow - and which we don't.</p> <h2 id="threat-modelling">Threat Modelling</h2> <p>To make an informed determination about what keys we should allow in our IDM we have to assess our individual risks. Generally we want to consider the damage from a compromised authenticator.</p> <p>When I'm inside my home network going to control my house lightbulbs, that's quite low risk. Private network, with little consequences of damage.</p> <p>For my personal email, compromise would represent a huge amount of personal damage as email is so critical to individual account security.</p> <p>If the release engineers in a linux distribution were compromised for example, that could lead to malware being distributed in the distribution aftecting thousands of people. Similar for developers of opensource where compromise of their credentials can lead to bugdoors being injected into code.</p> <h2 id="authenticator-risks">Authenticator Risks</h2> <p>Because of this, we need to consider the possible threat vectors against authenticators, and how an authenticator <em>may</em> become compromised.</p> <p>A problem here is that in Webauthn attestation, firmware versions are not (currently) included. This means that if any model or version of an authenticator's software or hardware is compromised, that <em>all</em> versions of that device must be considered compromised. This raises the bar for vendors to be sure they &quot;get it right&quot; when they ship a device the first time.</p> <h3 id="physical-attacks">Physical Attacks</h3> <p>This is the most obvious one. It can occur either in transit as the device is sent to you, or it can occur when the device is in your possesion.</p> <p>Mitigations are tamper evident packaging (transit only), tamper evident features on the device, or device construction that requires destruction actions to open.</p> <h3 id="local-software-attacks">Local Software Attacks</h3> <p>Rather than attacking the device physically, the device may be attacked locally via it's interfaces such as USB, NFC or Bluetooth. This leaves no physical evidence, but can still allow compromise of a device if physical access is obtained.</p> <p>Mitigations are tamper evident packaging (to prevent transit software attacks), and software hardening and use of memory safe languages.</p> <h3 id="remote-software-attacks">Remote Software Attacks</h3> <p>This is the most nefarious class, where a device can be compromised by it being connected to your machine as it visits a webpage. This is due to weaknesses in it's software via it's CTAP interface.</p> <p>Mitigations are software hardening, auditing for hidden vendor commands, and devices ensuring that any out of band management is never performed via the CTAP interface (it must be a seperate channel).</p> <h3 id="cryptographic-failures">Cryptographic Failures</h3> <p>This is a failure of the devices programming or hardware during cryptographic operations allowing private keys to be removed from the secure enclave that exists inside the device.</p> <p>Mitigations are in the hands of vendors requiring detailed audits and work to ensure their software is not vulnerable.</p> <h2 id="external-certification-bodies">External Certification Bodies</h2> <p>FIDO who certifies authenticators publishes a metadata service that shows details about various authenticators and their assessments. These can be found on the <a href="https://fidoalliance.org/metadata/">mds service site</a>.</p> <p>The MDS defines multiple certification levels for devices. These are defined <a href="https://fidoalliance.org/certification/authenticator-certification-levels/">on fido's site</a>.</p> <p>Currently the best reference for an overview is the <a href="https://fidoalliance.org/specs/fido-security-requirements/fido-authenticator-security-requirements-v1.5-fd-20211102.html#authenticator-hardware-examples">table of authenticator hardware examples</a>.</p> <p>We have a tool for querying this data <a href="https://github.com/kanidm/webauthn-rs/tree/master/fido-mds-tool">as part of webauthn-rs</a></p> <h2 id="what-authenticator-do-you-recommend">What Authenticator Do You Recommend?</h2> <p><a href="https://www.yubico.com/products/yubikey-5-overview/">Yubikey 5 Series</a></p> <p>These are the gold standard, and have all the best features here.</p> <ul> <li>Tamper evident packaging</li> <li>Hardware that can not be opened without destruction</li> <li>Hardened software to prevent local compromise</li> <li>No hidden vendor commands in CTAP preventing remote compromise</li> <li>Audits of their cryptographic processors to ensure they are high quality</li> <li>FIPS variant meets L2 FIDO criteria - the highest level currently granted by FIDO <ul> <li>Non FIPS variants are the same HW, so should be able to achieve L2 as well in future</li> </ul> </li> </ul> <p><a href="https://www.token2.com/shop/product/token2-t2f2-fido2-and-u2f-security-key">Token2 T2F2</a></p> <p>Runner up - this is the only other authenticator we have found with high quality tamper resitent features.</p> <ul> <li>No tamper evident packaging</li> <li>Hardware that can not be opened without destruction</li> <li>Hardened software to prevent local compromise</li> <li>No hidden vendor commands in CTAP preventing remote compromise</li> </ul> <h2 id="so-what-about-opensource-authenticators">So What About OpenSource Authenticators?</h2> <p>There are two major brands that I am aware of. Nitrokey and Solokey.</p> <p>Sadly both have issues, meaning I advise against their use.</p> <h3 id="solokey">Solokey</h3> <p>Solokey is the only opensource authenticator with a FIDO certification. They are built with an epoxy over their main circuits to highlight if these are altered. This is a great level of tamper resistence.</p> <p>The solokey is based on an LPC55S69 which unfortunately means it has been affected by a <a href="https://oxide.computer/blog/another-vulnerability-in-the-lpc55s69-rom?ref=brianlovin.com">trust chain bypass</a>. As a result NXP have had to issue new board revisions to resolve this, but it's not clear when you own a solokey if you have an affected revision or not. Since webauthn can't distinguish firmware versions either, it's also impossible to tell if a device has been fully software updated.</p> <p>In summary</p> <ul> <li>No tamper evident packaging</li> <li>Hardware that can not be opened without destruction</li> <li>Unable to tell if fully updated (vendor must change aaguids on fw version update)</li> <li>Local software trust chain bypass may be possible</li> </ul> <h3 id="nitrokey-3-nfc">Nitrokey 3 NFC</h3> <p>The Nitrokey 3 is built on the same hardware and software as the solokey. This means it is also vulnerable to the same issues. Unlike the solokey, the Nitrokey lacks tamper proof hardware - I can open mine quite easily with a spudger leaving no evidence of access. The NK3 I own is also a revision of the LPC55S69 which I can confirm is vulnerable to the trust chain bypass.</p> <ul> <li>No tamper evident packaging</li> <li>Hardware has no tamper resistance</li> <li>Unable to tell if fully updated (vendor must change aaguids on fw version update)</li> <li>Local software trust chain bypass may be possible</li> </ul> <h3 id="nitrokey-3-mini">Nitrokey 3 Mini</h3> <p>The Nitrokey 3 Mini uses the same software an the NK3N, but uses a different chip (nrf32). We are unable to assess the hardware security as we don't own one, but nitrokeys trackrecord would indicate it likely has poor tamper resistance.</p> <p>The <a href="https://shop.nitrokey.com/shop/nk3am-nitrokey-3a-mini-149?#attr=">NK3M shop page</a> as of 2023 declares:</p> <p><em>IMPORTANT NOTE: The included Secure Element is not used at this time. We are currently working on its integration and will enable its use later via firmware update.</em></p> <p>This means that keys are <em>very likely</em> not stored with secure mechanisms. Unless a firmware update that enables this feature changes the aaguid, it will always have to be assumed that this model of key is a version not using the secure enclave, making it inelligble for any serious use.</p> <ul> <li>No tamper evident packaging</li> <li>Hardware <em>likely</em> has no tamper resistance</li> <li>Unable to tell if fully updated (vendor must change aaguids on fw version update)</li> <li>Does not use devices own secure element</li> </ul> <h3 id="nitrokey-2">Nitrokey 2</h3> <p>Also called the Nitrokey FIDO2. While this device is named &quot;FIDO2&quot;, it has never been certified by FIDO. Like the NK 3, it has no tamper resistant packaging or hardware.</p> <p>The NK2 has a weird feature though - you can update it <em>from a website</em> ( <a href="https://update.nitrokey.com/">update site</a> ).</p> <p>The way this is performed is with a hidden set of vendor commands that are hidden inside the credential ID of the CTAP2 get assertion process, defined by the value <code>WALLET_TAG</code>. You can see the checks for this in <a href="https://github.com/Nitrokey/nitrokey-fido2-firmware/blob/12f2c14c6d7752293385c4c2f5317f16b0b63e61/fido2/extensions/extensions.c#L112">the nitrokey firmware source</a></p> <p>The problem here is that this means any website can sent a credential id with this <code>WALLET_TAG</code> and the nitrokey will respect and start to process those commands. More shocking, is these commands do not require interaction (touch) of the device or the pin to proceed, so visiting any webpage can trigger these commands. These commands include retrieving the device firmware versions, triggering the device to go into bootloader mode, and updating the firmware.</p> <p>At this time I am not aware of or sure about vulnerabilities in the NK2 L432KC6 chip. However, the ability to perform remote unauthenticated firmware updates is not something I want in a secure authenticator.</p> <p>Just like the other keys, since webauthn can't distinguish firmware versions either, it's also impossible to tell if a device has been fully software updated.</p> <ul> <li>No tamper evident packaging</li> <li>Hardware has no tamper resistance</li> <li>Unable to tell if fully updated (vendor must change aaguids on fw version update)</li> <li>Remote software trust chain bypass may be possible due to vendor commands</li> </ul> <h2 id="summary">Summary</h2> <p>Today there are no opensource keys that meet basic security requirements for use in higher security environments. This means that attestation of webauthn keys in these environments is critical to ensure that these and other compromised keys documented in the FIDO MDS are not used in these situations.</p> <p>If there are other models and makes of opensource security keys I haven't reviewed, let me know so that I can purchase them for auditing!</p> Getting Started with PKCS11 Tue, 14 Nov 2023 00:00:00 +0000 Unknown https://fy.blackhats.net.au/blog/2023-11-14-pkcs11-getting-started/ https://fy.blackhats.net.au/blog/2023-11-14-pkcs11-getting-started/ <h1 id="getting-started-with-pkcs11">Getting Started with PKCS11</h1> <p>PKCS11 is one of those horrible mystery technologies, that just seems to have no good starting place or reference on how to make it work. But it's also a technology that you see commonly around for hardware security modules (HSM), trusted platform modules (TPM) and other high impact cryptographic environments. This makes it an annoying chasm to cross for developers and administrators alike who want to configure these important tools for key security.</p> <p>So I decided to spend some time to learn about how this all works - scouring a variety of sources I hope to put together something that can help make it easier in future for others.</p> <h2 id="concepts">Concepts</h2> <p>PKCS11 is a specification defined by OASIS. The <a href="https://docs.oasis-open.org/pkcs11/pkcs11-spec/v3.1/cs01/pkcs11-spec-v3.1-cs01.html">specification</a> isn't very informative for a new user.</p> <p>What PKCS11 defines is an <em>abstraction</em> between an application and a security module. This allows the security module to be swapped with an alternative and the application requires no changes.</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>┌─────────────────┐ </span><span>│ │ </span><span>│ Application │ </span><span>│ │ </span><span>└─────────────────┘ </span><span> │ </span><span> ▼ </span><span>┌─────────────────┐ </span><span>│ PKCS11 Aware │ </span><span>│ Cryptographic │ </span><span>│ Library │ </span><span>└─────────────────┘ </span><span> │ </span><span> Cryptoki API </span><span> │ </span><span> ▼ </span><span>┌─────────────────┐ </span><span>│ │ </span><span>│ PKCS11 Module │ </span><span>│ │ </span><span>└─────────────────┘ </span><span> │ </span><span> ▼ </span><span>┌─────────────────┐ </span><span>│ │ </span><span>│ HSM │ </span><span>│ │ </span><span>└─────────────────┘ </span></code></pre> <p>Commonly PKCS11 modules will provide an abstraction for an HSM or a TPM, but there are also software HSM that are useful for testing. This also means that if you have a vendor who provides you a HSM they will commonly also provide you a PKCS11 module to allow your application to communicate with it.</p> <p>The PKCS11 module itself is just a dynamic library (so, dll, or dylib) which implements and exposes the Cryptoki C API. This is how an application can dynamically load the module and use it with no changes.</p> <p>Since this is an abstraction it also means that the cryptographic material stored in our HSM has a standard layout and structure that applications can expect.</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span> ┌─────────────────┐ </span><span> ┌─┴───────────────┐ │ </span><span> ┌─────────────────┐ ┌┴────────────────┐│ │ </span><span> │ │ │ ││ │ </span><span> ┌───▶│ Token / Slot │───▶│ Object │├─┘ </span><span> │ │ │ │ ├┘ </span><span>┌─────────────────┐ │ └─────────────────┘ └─────────────────┘ </span><span>│ │ │ </span><span>│ PKCS11 Module │───┤ ┌─────────────────┐ </span><span>│ │ │ ┌─┴───────────────┐ │ </span><span>└─────────────────┘ │ ┌─────────────────┐ ┌┴────────────────┐│ │ </span><span> │ │ │ │ ││ │ </span><span> └───▶│ Token / Slot ├───▶│ Object │├─┘ </span><span> │ │ │ ├ </span><span> └─────────────────┘ └─────────────────┘ </span></code></pre> <p>Each module has a number of tokens (also called slots). Then for each token, these have many objects associated which can be private keys, public keys, x509 certificates and more.</p> <p>Authentication is performed from the application to the token. There are two authentication values: the security officer PIN, and the user PIN. Generally this allows separation between the ability to write/destroy objects, and the ability to use them in an application.</p> <h2 id="prerequisites">Prerequisites</h2> <p>The easiest way to &quot;test&quot; this is with a virtual machine configured with a softtpm. You can add this in libvirt with the xml:</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>&lt;domain type=&#39;kvm&#39;&gt; </span><span> &lt;devices&gt; </span><span> &lt;tpm model=&#39;tpm-crb&#39;&gt; </span><span> &lt;backend type=&#39;emulator&#39; version=&#39;2.0&#39;/&gt; </span><span> &lt;/tpm&gt; </span><span> &lt;/devices&gt; </span><span>&lt;/domain&gt; </span></code></pre> <p>If using virt-manager you can add a tpm under the add hardware menu. Ensure you set this to CRB (command request buffer) and version 2.0.</p> <p>If you are using OpenSUSE you will need the following packages:</p> <ul> <li>tpm2-pkcs11 - the module exposing TPMs via the cryptoki api</li> <li>opensc - a library for interacting with and administering pkcs11 modules</li> <li>openssl-3 - generic cryptographic library</li> <li>pkcs11-provider - a pkcs11 provider for openssl-3 allowing it to use our pkcs11 modules</li> </ul> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>zypper install openssl-3 pkcs11-provider tpm2-pkcs11 opensc </span></code></pre> <p>This certainly isn't an exhaustive or limited list - there are plenty of other tools that can interact with pkcs11, for example p11-kit.</p> <h2 id="setting-up-our-device">Setting Up Our Device</h2> <p>pkcs11 modules are generally installed into <code>/usr/lib64/pkcs11</code>. opensc does not have a way to list the module that are available. (p11-kit does with <code>p11-kit list-modules</code>)</p> <p>For our example we're going to use the tpm pkcs11 provider at <code>/usr/lib64/pkcs11/libtpm2_pkcs11.so</code></p> <p>We can show info about the module with:</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>pkcs11-tool --module /usr/lib64/pkcs11/libtpm2_pkcs11.so --show-info </span></code></pre> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>Cryptoki version 2.40 </span><span>Manufacturer tpm2-software.github.io </span><span>Library TPM2.0 Cryptoki (ver 1.9) </span><span>Using slot 0 with a present token (0x1) </span></code></pre> <p>We can now list all the slots with:</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>pkcs11-tool --module /usr/lib64/pkcs11/libtpm2_pkcs11.so --list-slots </span></code></pre> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>Available slots: </span><span>Slot 0 (0x1): </span><span> token state: uninitialized </span></code></pre> <p>We can see that we have a single slot and it's not initialised yet. So lets initialise it. You'll notice that we use &quot;--slot 1&quot; even though the output above says &quot;Slot 0&quot;. This is because we are using the slot id in hex. Off by one errors are everywhere :)</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>pkcs11-tool --module /usr/lib64/pkcs11/libtpm2_pkcs11.so --init-token --slot 1 --label dev </span></code></pre> <p>You should be prompted to setup the security office (SO) PIN. At this point we can list our slots again and see:</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>Available slots: </span><span>Slot 0 (0x1): dev </span><span> token label : dev </span><span> token manufacturer : IBM </span><span> token model : SW TPM </span><span> token flags : login required, rng, token initialized, PIN initialized </span><span> hardware version : 1.62 </span><span> firmware version : 25.35 </span><span> serial num : 0000000000000000 </span><span> pin min/max : 0/128 </span><span>Slot 1 (0x2): </span><span> token state: uninitialized </span></code></pre> <p>The number of slots will keep growing as we add more slots.</p> <blockquote> <p>TIP: <code>libtpm2_pkcs11</code> is actually storing it's keys in an sqlite database on your system and dynamically loading and unloading them from the TPM as required. These keys are stored sealed with a TPM internal key so that only this TPM can use the keys. This means you don't have to worry about running out of space on your TPM.</p> </blockquote> <p>We can now also setup our user pin, using the SO to trigger the PIN change.</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>pkcs11-tool --module /usr/lib64/pkcs11/libtpm2_pkcs11.so --slot 1 --login --login-type so --init-pin </span></code></pre> <h2 id="managing-objects">Managing Objects</h2> <p>We can show the objects in our slot with:</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>pkcs11-tool --module /usr/lib64/pkcs11/libtpm2_pkcs11.so --login --slot 1 --list-objects </span></code></pre> <p>Initially this output should be empty - we haven't made anything yet!</p> <p>We can create a new ECDSA key. This key will have a label and an ID associated so that we can reference the key uniquely.</p> <blockquote> <p>NOTE: If you don't set --id here, then some pkcs11 providers will fail to use your key. Ensure you set both a label and an ID!</p> </blockquote> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>pkcs11-tool --module /usr/lib64/pkcs11/libtpm2_pkcs11.so --login --slot 1 --keypairgen \ </span><span> --key-type EC:prime256v1 --label &quot;my_key&quot; --usage-sign --id 01 </span></code></pre> <p>Now we can see our objects with <code>list-objects</code></p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>Public Key Object; EC EC_POINT 256 bits </span><span> EC_POINT: 04410430bb3c90d2caae1a848e99e3cfb265bfcc969274d478a57050fff94749cc04c23e84aa11a86b500d0c888da331c20abecf8cc1b4ed15ef5535561f0678cc580f </span><span> EC_PARAMS: 06082a8648ce3d030107 </span><span> label: my_key </span><span> ID: 01 </span><span> Usage: encrypt, verify </span><span> Access: local </span><span>Private Key Object; EC </span><span> label: my_key </span><span> ID: 01 </span><span> Usage: decrypt, sign </span><span> Access: sensitive, always sensitive, never extractable, local </span><span> Allowed mechanisms: ECDSA,ECDSA-SHA1,ECDSA-SHA256,ECDSA-SHA384,ECDSA-SHA512 </span></code></pre> <p>We have two objects - An ECDSA public key, and the ECDSA private key. We can also see what the key can be used for and allowed algorithms. Finally we have some attributes that defined properties of the key.</p> <ul> <li>local - the content of the key was generated inside a HSM</li> <li>sensitive - means that the content of the key can not be disclosed in plaintext</li> <li>always sensitive - the key has never been disclosed outside of a HSM</li> <li>extractable - the key can be extracted from this device in an encrypted manner</li> <li>never extractable - the key has never been extracted from this device</li> </ul> <p>These properties allow us to see useful things:</p> <ul> <li>A local, always sensitive and never extractable key shows us that this key was generated and used only in this HSM</li> <li>A sensitive only key that is extractable may have been shared between other HSMs with a wrapping key</li> <li>A sensitive key that does NOT have the always sensitive and local properties was likely imported from an external source</li> </ul> <p>If we wanted now we could delete these objects:</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>pkcs11-tool --module /usr/lib64/pkcs11/libtpm2_pkcs11.so --login --slot 1 --delete-object \ </span><span> --label my_key --type privkey </span><span>pkcs11-tool --module /usr/lib64/pkcs11/libtpm2_pkcs11.so --login --slot 1 --delete-object \ </span><span> --label my_key --type pubkey </span></code></pre> <p>Notice we have to do this twice? Once for the public key, once for the private key.</p> <h2 id="using-our-keys-with-openssl">Using Our Keys with OpenSSL</h2> <p>At this point we may wish to use our key with openssl, perhaps we want to request a certificate to be signed or some other operation.</p> <p>First we need to configure openssl to understand where our pkcs11 modules are:</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span># provider.conf </span><span>openssl_conf = openssl_init </span><span> </span><span>[openssl_init] </span><span>providers = provider_sect </span><span> </span><span>[provider_sect] </span><span>default = default_sect </span><span>pkcs11 = pkcs11_sect </span><span> </span><span>[default_sect] </span><span>activate = 1 </span><span> </span><span>[pkcs11_sect] </span><span>module = /usr/lib64/ossl-modules/pkcs11.so </span><span>pkcs11-module-path = /usr/lib64/pkcs11/libtpm2_pkcs11.so </span><span>activate = 1 </span></code></pre> <p>We can check this worked with:</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>OPENSSL_CONF=provider.conf openssl list -providers -provider pkcs11 </span></code></pre> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>Providers: </span><span> default </span><span> name: OpenSSL Default Provider </span><span> version: 3.1.4 </span><span> status: active </span><span> pkcs11 </span><span> name: PKCS#11 Provider </span><span> version: 3.1.3 </span><span> status: active </span></code></pre> <p>We can now show details about our key with:</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>OPENSSL_CONF=provider.conf openssl pkey -provider pkcs11 -noout -text \ </span><span> -in &quot;pkcs11:token=dev;label=my_key&quot; </span></code></pre> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>PKCS11 EC Private Key (256 bits) </span><span>[Can&#39;t export and print private key data] </span><span>URI pkcs11:model=SW%20%20%20TPM;manufacturer=IBM;serial=0000000000000000;token=dev;id=%01;object=my_key;type=private </span></code></pre> <p>You'll notice a new concept here - pkcs11 supports URI's to reference specific objects. This allows you to have many providers, with many slots and objects, and still uniquely interact with specific objects.</p> <p>As you can see from our URI of <code>pkcs11:token=dev;label=my_key</code> we are referencing our key by its token name <code>dev</code> and the <code>label</code> we gave it earlier during key creation. The openssl display shows a number of other possible elements in the URI that we could use. For a full list see <a href="https://www.rfc-editor.org/rfc/rfc7512#section-2.3">rfc7512 section 2.3</a>.</p> <p>At this point openssl can see our key and so now we can make a CSR.</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>OPENSSL_CONF=provider.conf openssl req -provider pkcs11 \ </span><span> -new -days 1 -subj &#39;/CN=my_key/&#39; -out csr.pem </span><span> -key &quot;pkcs11:token=dev;label=my_key&quot; </span></code></pre> <p>Once this is signed we can load this into our pkcs11 module:</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>pkcs11-tool --module /usr/lib64/pkcs11/libtpm2_pkcs11.so --write-object cert.pem \ </span><span> --label &quot;my_key&quot; --id 01 --type cert --login --login-type so </span></code></pre> <p>Notice we use the same label and id as the key and public key? This way the URI only needs to change to specify the type of the object to find the correct associated key and public key.</p> <p>Now when we list our objects we can see:</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>Public Key Object; EC EC_POINT 256 bits </span><span> EC_POINT: 04410430bb3c90d2caae1a848e99e3cfb265bfcc969274d478a57050fff94749cc04c23e84aa11a86b500d0c888da331c20abecf8cc1b4ed15ef5535561f0678cc580f </span><span> EC_PARAMS: 06082a8648ce3d030107 </span><span> label: my_key </span><span> ID: 01 </span><span> Usage: encrypt, verify </span><span> Access: local </span><span>Private Key Object; EC </span><span> label: my_key </span><span> ID: 01 </span><span> Usage: decrypt, sign </span><span> Access: sensitive, always sensitive, never extractable, local </span><span> Allowed mechanisms: ECDSA,ECDSA-SHA1,ECDSA-SHA256,ECDSA-SHA384,ECDSA-SHA512 </span><span>Certificate Object; type = X.509 cert </span><span> label: my_key </span><span> subject: DN: CN=my_key </span><span> serial: 57812D09A3503723F038BA407CD7D2E965289FC1 </span><span> ID: 01 </span></code></pre> <h2 id="application-integration">Application Integration</h2> <p>Once we have our key and certificate in the TPM, most applications that use openssl require very little modification.</p> <ul> <li>Ensure that we have the <code>OPENSSL_CONF=</code> with the pkcs11 provider in the environment of the application</li> <li>Change the private key file to a pkcs11 url</li> </ul> <p>Over time I'll add more examples here as I work on them.</p> <h2 id="using-a-wrap-key-to-move-objects">Using a Wrap Key to Move Objects</h2> <p>TBD</p> <h2 id="debugging">DEBUGGING</h2> <p>As with anything, these tools can break and we need to be able to debug them.</p> <ul> <li><code>TPM2_PKCS11_LOG_LEVEL=1</code> - set the log level of the <code>tpm2_pkcs11</code> module. Values are 0, 1, 2</li> <li><code>PKCS11_PROVIDER_DEBUG=file:/dev/stderr,level:1</code> - set the log level of the openssl pkcs11 provider. Values are 0, 1, 2</li> </ul> SSH Key Storage Comparison Tue, 24 Oct 2023 00:00:00 +0000 Unknown https://fy.blackhats.net.au/blog/2023-10-24-ssh-key-storage-comparisons/ https://fy.blackhats.net.au/blog/2023-10-24-ssh-key-storage-comparisons/ <h1 id="ssh-key-storage">SSH Key Storage</h1> <p>A kind reader asked me an interesting question the other day. &quot;What do you think of the choice of ssh sk keys between ecdsa and ed25519?&quot;. At the same time, within Kanidm we have actually been discussing the different approaches we could take with ssh key handling in the future between ssh cas and ssh sk key attestation, especially once we consider service accounts.</p> <p>As with anything in security we always need to balance the technology with the risks and threats that we are trying to mitigate or prevent.</p> <p>Compared to the use of a username and password, using ssh keys to authenticate to a remote server is significantly more secure. This is especially true if you disable password authentication methods.</p> <h2 id="cryptographic-considerations">Cryptographic Considerations</h2> <p>Of course, just using cryptographic authentication isn't free. While we no longer have the risks of passwords, the threats that exist against cryptographic keys are quite different. This means we now need to ensure that we follow and keep up to date with developments in crytpgraphy.</p> <p>There are many sites that can assist with key requirements such as <a href="https://www.keylength.com">keylength.com</a>. The following table was generated on 2023-09-25 with the year set to 2030 (meaning these key sizes values are <em>predicted</em> to be safe until 2030).</p> <blockquote> <table><thead><tr><th>Method</th><th>Date</th><th>Symmetric</th><th>FM</th><th>DL Key</th><th>DL Group</th><th>Elliptic Curve</th><th>Hash</th></tr></thead><tbody> <tr><td>Lenstra / Verheul</td><td>2030</td><td>93</td><td>2493^2016</td><td>165</td><td>2493</td><td>176</td><td>186</td></tr> <tr><td>Lenstra Updated </td><td>2030</td><td>88</td><td>1698^2063</td><td>176</td><td>1698</td><td>176</td><td>176</td></tr> <tr><td>ECRYPT</td><td>2029-2068</td><td>256</td><td>15360</td><td>512</td><td>15360</td><td>512</td><td>512</td></tr> <tr><td>NIST</td><td>2019-2030</td><td>112</td><td>2048</td><td>224</td><td>2048</td><td>224</td><td>224</td></tr> <tr><td>ANSSI</td><td>&gt; 2030</td><td>128</td><td>3072</td><td>200</td><td>3072</td><td>256</td><td>256</td></tr> <tr><td>NSA</td><td>-</td><td>256</td><td>3072</td><td>-</td><td>-</td><td>384</td><td>384</td></tr> <tr><td>RFC3766 </td><td>-</td><td>-</td><td>-</td><td>-</td><td>-</td><td>-</td><td>-</td></tr> <tr><td>BSI</td><td>-</td><td>-</td><td>-</td><td>-</td><td>-</td><td>-</td><td>-</td></tr> </tbody></table> <ul> <li>DL - Discrete Logarithm</li> <li>FM - Factoring Modulus</li> </ul> </blockquote> <p>The values here that are important for ssh key generation is the Factoring Modulus (FM) and Elliptic Curve.Each of these sources is assuming different risks and requirements. You will need to make your own informed choices about these but reading between the values here your minimum key size should be at least RSA 2048 and ECDSA P256 / ED25519, or if you wish to future proof you may wish to choose RSA 3072, ECDSA P512 / ED448.</p> <h2 id="p-256-vs-ed25519">P-256 vs ED25519</h2> <p>Our dear reader asked if you should choose ED25519 over ECDSA P-256 as there are two attacks (LadderLeak, Minerva) that affect ECDSA. I am not a cryptographer, so my view here should be taken with caution, but reading both papers the authors indicate that these attacks require a large number of signatures to conduct meaning that outside of research conditions, these may not be viable against ssh keys. The authors themself validate that these attacks may not be possible outside of research conditions.</p> <h2 id="ssh-key-storage-1">SSH Key Storage</h2> <p>Since we now have cryptographic keys for authentication we need to care about how we store these. Storage of keys is just as important as the type of key we use.</p> <h3 id="files-on-disk">Files on Disk</h3> <p>Most peoples first introduction to ssh keys is storing them as files in their home directory. If during the generation you entered a passphrase, then this stores the private keys as encrypted files. If you do not specify a passphrase these are stored unencrypted. Generally it's a good idea to store these with a passphrase so that theft of the private key means an attacker can not immediately use this for nefarious purposes.</p> <h3 id="security-keys">Security Keys</h3> <p>Rather than storing ssh keys in files, they can be generated in the secure enclave of a security key such as a yubikey. Because these are stored in the secure enclave, malware or an attacker can not take the keys from the machine. Additionally, because of how these keys work, they will not operate without physical interaction with the key.</p> <p>Another unique property of these keys is they can strictly enforce that userverification (such as a PIN or biometric) is validated as well as physical interaction before they can proceed. This verification requirement is baked into the key at creation time and enforced by the secure unclave so that it can not be bypassed, adding an extra level of security - changing the key from something you have to something you have and are/know.</p> <p>These keys can also attest (cryptographically prove) that they are in a secure enclave at creation time, so that for higher security environments they can assert that keys <em>must</em> be hardware backed.</p> <p>So far we have mentioned these keys are in the secure enclave. By default the actual private key is stored in a file, encrypted by a secret master key inside the security key. This is what allows these keys to have &quot;unlimited storage&quot; as keys are loaded and unloaded as needed (the same way that a TPM works).</p> <p>However for the most secure environments, they may wish that the key never leaves the enclave even if in a secure encrypted form. Security keys have extremely limited storage (generally 8 keys, some models up to 25), so this limits the use of these resident keys. But if they are required, you can create a resident, attested, user verified key for the highest levels of assurance.</p> <h3 id="certificate-authorities">Certificate Authorities</h3> <p>ssh keys normally rely on the public key being transferred to the machine to authenticate too. This leads to strategies like ssh key distribution in kanidm so that a central server can be consulted for which authorised keys are valid for authentication.</p> <p>However a ssh certificate authority functions more like TLS where a certificate authority's key is trusted by the servers, and then users are issued an ssh key <em>signed</em> by that authority. When authenticating to the server since the user certificate is signed by a trusted authority it can be allowed to proceed.</p> <p>Since these keys are issued as files they carry some of the same risks as our previous files. However because there is an authority that can issue the keys, they can be created with a short expiration as required. This leads to some interesting configurations where an external tool can be used to issue certificates as required, limited to specific hosts and commands.</p> <h2 id="comparison">Comparison</h2> <p>As with anything, all of these approaches have pros and cons. It's up to you to decide what will be best in your scenario.</p> <blockquote> <table><thead><tr><th>Key Type</th><th style="text-align: center">Strict Enforcement</th><th style="text-align: center">Exfiltration Possible</th><th style="text-align: center">Expiration</th><th style="text-align: center">Hardware Bound</th><th style="text-align: center">User Verification</th></tr></thead><tbody> <tr><td>ssh key</td><td style="text-align: center">no</td><td style="text-align: center">yes</td><td style="text-align: center">no</td><td style="text-align: center">no</td><td style="text-align: center">no</td></tr> <tr><td>encrypted ssh key</td><td style="text-align: center">no</td><td style="text-align: center">yes</td><td style="text-align: center">no</td><td style="text-align: center">no</td><td style="text-align: center">yes-ish</td></tr> <tr><td>sk key</td><td style="text-align: center">no</td><td style="text-align: center">no</td><td style="text-align: center">no</td><td style="text-align: center">yes</td><td style="text-align: center">yes</td></tr> <tr><td>attested sk key</td><td style="text-align: center">yes</td><td style="text-align: center">no</td><td style="text-align: center">no</td><td style="text-align: center">yes</td><td style="text-align: center">yes</td></tr> <tr><td>ca key</td><td style="text-align: center">yes</td><td style="text-align: center">yes</td><td style="text-align: center">yes</td><td style="text-align: center">possible</td><td style="text-align: center">yes*</td></tr> </tbody></table> </blockquote> <p>* If you use ca keys with sk keys, or your issuing ca provides a form of verification</p> <p>For some example use cases:</p> <p><em>I am a home user logging into my server</em></p> <ul> <li>Use what ever makes you happy</li> <li>Any ssh key auth is better than a password</li> </ul> <p><em>I am logging into production servers</em></p> <ul> <li>Use ssh key files with passphrases</li> <li>sk keys with enforce user verification</li> </ul> <p><em>I am a business looking to enforce secure keys for admins/developers</em></p> <ul> <li>Use attested sk keys with enforced user verification</li> <li>Deploy an ssh ca that requires authentication to issue certificates</li> </ul> <p><em>I am a three letter agency that ...</em></p> <ul> <li>Ask your security team, that's what they're there for.</li> <li>But also, use attested resident sk keys with enforced user verification</li> </ul> <p>There is no perfect answer here, but you need to consider the risks you face and how you want to mitigate them. The biggest problem of all of this is <em>proof</em>. Once you move from &quot;I want an ssh key&quot; to &quot;we want to enforce our requirements&quot; this adds extra challenges that only an ssh ca or attested sk keys can fufil. While it's nice to trust our users, strictly enforced requirements are far better when it comes to security.</p> <h2 id="things-not-to-do">Things Not To Do</h2> <h3 id="sshfp-dns-records">SSHFP DNS Records</h3> <p>Using SSHFP DNS records is insecure. This is because even if you have DNSSEC, it <a href="https://sockpuppet.org/blog/2015/01/15/against-dnssec/">does nothing to protect you.</a></p> <p>This approaches leaves you open to MITM attacks which is effectively a path to remote unauthorised access.</p> <h3 id="ssh-key-distribution-with-ldap-starttls">SSH Key Distribution with LDAP StartTLS</h3> <p>When distributing keys with LDAP, you must always use LDAPS. This is because <a href="/2021/08/12/starttls_in_ldap.html">LDAP with StartTLS is insecure.</a></p> <h2 id="ssh-key-creation-reference">SSH Key Creation Reference</h2> <h3 id="files-on-disk-1">Files on Disk</h3> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>ssh-keygen -t [rsa | ecdsa | ed25519] -b &lt;bits&gt; </span><span>ssh-keygen -t ed25519 </span><span>ssh-keygen -t ecdsa -b 512 </span><span>ssh-keygen -t rsa -b 3072 </span></code></pre> <h3 id="security-keys-basic">Security Keys (Basic)</h3> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>ssh-keygen -t [ecdsa-sk | ed25519-sk] </span><span>ssh-keygen -t ecdsa-sk </span><span>// Note: Not all keys support ed25519 </span><span>ssh-keygen -t ed25519-sk </span></code></pre> <h3 id="security-keys-user-verified">Security Keys (User Verified)</h3> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>ssh-keygen -t [ecdsa-sk | ed25519-sk] -O verify-required </span><span>ssh-keygen -t ecdsa-sk -O verify-required </span></code></pre> <h3 id="certificate-authority">Certificate Authority</h3> Storage Administration Guide Fri, 08 Sep 2023 00:00:00 +0000 Unknown https://fy.blackhats.net.au/pages/storage-admin-guide/ https://fy.blackhats.net.au/pages/storage-admin-guide/ <h1 id="storage-administration-guide">Storage Administration Guide</h1> <p>This guide will help you understand, configure and maintain storage on Linux servers. The content of this guide is optimised for reliability and accesibility. This is based not only on my own experiences but observing the experiences of enterprise customers for many years.</p> <h2 id="warning-warnings-warning">⚠️ Warnings ⚠️</h2> <p>Making changes to storage entails risks. Linux and it's storage tools have no safety barriers. Mistakes can result in <em>COMPLETE LOSS OF ALL YOUR DATA</em>.</p> <p><em>DO NOT COPY PASTE COMMANDS HERE WITHOUT UNDERSTANDING THEM.</em></p> <p><em>CAREFULLY PLAN YOUR COMMANDS.</em></p> <p><em>HAVE BACKUPS THAT YOU HAVE TESTED.</em></p> <p>Almost all commands in this document require root privilieges.</p> <p>This document is a work in progress!</p> <h2 id="general-advice">General Advice</h2> <p>Before executing commands that will change your storage you should analyse your storage, make notes, and prepare your commands in a notepad before you execute the commands. This will allow you to review before making changes.</p> <h2 id="understanding-your-storage">Understanding Your Storage</h2> <p>Before changing your storage configurations you need to understand what you have and what you may want to achieve.</p> <h3 id="list-storage-on-your-system">List Storage On Your System</h3> <p><code>lsblk</code> is the most important command in your toolbox. It allows you to understand the layout and set of your storage between making changes. It also allows you to check which disks are in use so that you can <em>avoid</em> them while making changes.</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span># lsblk </span><span>NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINTS </span><span>sr0 11:0 1 372K 0 rom </span><span>vda 254:0 0 10G 0 disk </span><span>├─vda1 254:1 0 2M 0 part </span><span>├─vda2 254:2 0 33M 0 part /boot/efi </span><span>└─vda3 254:3 0 10G 0 part / </span><span>vdb 254:16 0 50G 0 disk </span><span>vdc 254:32 0 50G 0 disk </span><span>vdd 254:48 0 50G 0 disk </span><span>vde 254:64 0 50G 0 disk </span></code></pre> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span># lsblk </span><span>NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINTS </span><span>sr0 11:0 1 372K 0 rom </span><span>vda 254:0 0 10G 0 disk &lt;---------- this is a whole disk. </span><span> /-- these partitions exist on the disk. </span><span>├─vda1 254:1 0 2M 0 part - &lt;- this partition is not mounted </span><span>├─vda2 254:2 0 33M 0 part /boot/efi | &lt;- this partition is mounted at /boot/efi </span><span>└─vda3 254:3 0 10G 0 part / - &lt;- this partition is mounted on / </span><span>vdb 254:16 0 50G 0 disk </span><span>vdc 254:32 0 50G 0 disk &lt;---------- these disks have no partitions </span><span>vdd 254:48 0 50G 0 disk </span><span>vde 254:64 0 50G 0 disk </span></code></pre> <h3 id="view-filesystems-that-mount-at-boot">View Filesystems That Mount At Boot</h3> <p>Filesystems are mounted at boot from the <em>F</em>ile<em>S</em>ystem <em>TAB</em>le. These are stored in <code>/etc/fstab</code>.</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span># cat /etc/fstab </span><span>UUID=ef76b8e7-6017-4757-bd51-3e0e662d408b / xfs defaults 0 1 </span><span>UUID=F6F5-05FB /boot/efi vfat defaults 0 0 </span></code></pre> <p>This is arranged as a white-space separted table.</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>/- The path or identifier of the device to mount </span><span>| </span><span>| /- where to mount the filesystem </span><span>| | </span><span>| | /- the type of filesysetm on the device </span><span>| | | </span><span>| | | /- mount options for the filesystem to be mounted </span><span>| | | | </span><span>| | | | /- dump to tape on crash. set to 0 </span><span>| | | | | </span><span>| | | | | /- order of filesystem checks at boot. </span><span>| | | | | | 0 = do not check </span><span>| | | | | | 1 = check first </span><span>| | | | | | 2 = check second ... </span><span>v v v v v v </span><span>UUID=ef76b8e7-6017-4757-bd51-3e0e662d408b / xfs defaults 0 1 </span><span>/dev/disk/by-id/wwn-0x5001b448bd8e7de2 /mnt xfs defaults 0 0 </span></code></pre> <h3 id="show-disk-path-or-identifiers">Show Disk Path Or Identifiers</h3> <p>Linux allows disks to be referenced by different aliases that can be more accessible or unique to help prevent mistakes. For example, <code>vda</code> and <code>sda</code> are very similar but <code>virtio-pci-0000:04:00.0</code> or <code>wwn-0x5000cca0bbefc231</code> are distinct and uniquely identify the device. In addition if you move the device between different ports (e.g. changing sata ports, or sas ports) then some of these identifiers will stay stable between those changes.</p> <h4 id="show-disks-by-identifiers">Show disks by identifiers</h4> <p>Disk by ID are a stable identifier that should not change between systems or connection of the device.</p> <p>These are commonly used in ZFS pools or administration commands.</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span># ls -l /dev/disk/by-id </span><span>lrwxrwxrwx 1 root root 9 Sep 7 08:51 scsi-1ATA_WDC_WDS200T1R0A-68A4W0_223609A005A5 -&gt; ../../sdg </span><span>lrwxrwxrwx 1 root root 10 Sep 7 08:51 scsi-1ATA_WDC_WDS200T1R0A-68A4W0_223609A005A5-part1 -&gt; ../../sdg1 </span><span>lrwxrwxrwx 1 root root 10 Sep 7 08:51 scsi-1ATA_WDC_WDS200T1R0A-68A4W0_223609A005A5-part9 -&gt; ../../sdg9 </span><span>... </span><span>lrwxrwxrwx 1 root root 9 Sep 7 08:51 wwn-0x5001b448bd8e7de2 -&gt; ../../sdg </span><span>lrwxrwxrwx 1 root root 10 Sep 7 08:51 wwn-0x5001b448bd8e7de2-part1 -&gt; ../../sdg1 </span><span>lrwxrwxrwx 1 root root 10 Sep 7 08:51 wwn-0x5001b448bd8e7de2-part9 -&gt; ../../sdg9 </span></code></pre> <h4 id="show-disks-by-uuid">Show disks by UUID</h4> <p>Generally only partitions with filesystems will have a UUID - this will not show &quot;devices&quot;.</p> <p>UUID's are a stable identifier that should not change between systems or connection of the device.</p> <p>These are commonly used in <code>/etc/fstab</code> for mounting filesystems. These are used with the <code>fstab</code> syntax of <code>UUID=&lt; ... &gt;</code></p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span># ls -l /dev/disk/by-uuid </span><span>lrwxrwxrwx 1 root root 15 Sep 7 08:51 DA2C-4E2B -&gt; ../../nvme1n1p1 </span><span>lrwxrwxrwx 1 root root 10 Sep 7 08:51 e3967de9-6cab-4387-8a58-aa6b34dba39f -&gt; ../../dm-9 </span></code></pre> <h4 id="show-disks-by-their-physical-attachment-path">Show disks by their physical attachment path</h4> <p>These are commonly used to locate the type of device, and where it may be attached on a system.</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span># ls -l /dev/disk/by-path </span><span>lrwxrwxrwx 1 root root 9 Sep 7 08:51 pci-0000:00:17.0-ata-8.0 -&gt; ../../sdg </span><span>lrwxrwxrwx 1 root root 10 Sep 7 08:51 pci-0000:00:17.0-ata-8.0-part1 -&gt; ../../sdg1 </span><span>lrwxrwxrwx 1 root root 10 Sep 7 08:51 pci-0000:00:17.0-ata-8.0-part9 -&gt; ../../sdg9 </span><span>lrwxrwxrwx 1 root root 13 Sep 7 08:51 pci-0000:01:00.0-nvme-1 -&gt; ../../nvme0n1 </span><span>lrwxrwxrwx 1 root root 15 Sep 7 08:51 pci-0000:01:00.0-nvme-1-part1 -&gt; ../../nvme0n1p1 </span><span>lrwxrwxrwx 1 root root 15 Sep 7 08:51 pci-0000:01:00.0-nvme-1-part2 -&gt; ../../nvme0n1p2 </span><span>lrwxrwxrwx 1 root root 15 Sep 7 08:51 pci-0000:01:00.0-nvme-1-part3 -&gt; ../../nvme0n1p3 </span></code></pre> <h2 id="what-storage-setup-do-i-want">What Storage Setup Do I Want?</h2> <p>Here are some questions that may help you to decide how to configure the storage in your system.</p> <h3 id="i-m-installing-my-favourite-distro-on-a-laptop-workstation">I'm Installing My Favourite Distro On A Laptop/Workstation</h3> <ul> <li>You should use GPT for partitioning.</li> <li>You should use LVM to allow resizing partitions, creating raid or changing disks in the future.</li> <li>Split Home vs Root OR combined Home + Root is a personal preference.</li> <li>If you want fast and highly reliable storage -&gt; Choose XFS</li> <li>If you want features like snapshots -&gt; Choose BTRFS</li> </ul> <h3 id="i-m-adding-non-root-disks-to-my-workstation-server">I'm Adding Non-Root Disks To My Workstation/Server</h3> <h4 id="i-want-highly-reliable-fault-tolerant-storage">I Want Highly Reliable, Fault Tolerant Storage</h4> <ul> <li>Use ZFS</li> </ul> <h4 id="i-can-not-afford-to-lose-data-ever">I Can Not Afford To Lose Data Ever.</h4> <ul> <li>Use ZFS</li> </ul> <h4 id="i-want-one-giant-pool-of-storage-that-i-will-expand-in-future">I Want One Giant Pool Of Storage That I Will Expand In Future</h4> <ul> <li>Use ZFS</li> </ul> <h4 id="i-plan-to-add-remove-change-disks-again-when-ever-i-feel-like">I Plan To Add/Remove/Change Disks Again When Ever I Feel Like</h4> <ul> <li>Use LVM+RAID</li> </ul> <h4 id="i-just-want-disks-mounted-like-a-chaos-goblin">I Just Want Disks Mounted Like A Chaos Goblin</h4> <ul> <li>You should use GPT for partitioning.</li> <li>You should use LVM to allow resizing partitions, creating raid or changing disks in the future.</li> <li>If you want fast and highly reliable storage -&gt; Choose XFS</li> <li>If you want features like snapshots -&gt; Choose BTRFS</li> </ul> <h2 id="managing-single-disks">Managing Single Disks</h2> <h3 id="reload-partition-tables">Reload partition tables</h3> <p>After making changes to partition tables, you may need to force the kernel to re-read them so that they are reflected in commands like <code>lsblk</code></p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span># partprobe </span></code></pre> <h2 id="zfs">ZFS</h2> <p>todo!();</p> <h2 id="lvm">LVM</h2> <p>LVM is a Logical Volume Manager for Linux. It allows dynamically changing storage without downtime, and the ability to warp and shape into complex and weird disk layouts and geometries. It has excellent observability into the state of storage, and you should consider it a &quot;must have&quot; on all systems.</p> <p>LVM's single limitation is you can not use it for <code>/boot</code> or <code>EFI System Partitions</code>. These must remain as &quot;true&quot; partitions. If in doubt, trust your installer.</p> <p>LVM combines a set of physical volumes (PV) into a volume group (VG). A volume group contains a pool of available storage. logical volumes (LV) can then be created within that volume group that consume that storage. Each logical volume can have it's own characteristics like raid levels. Logical volumes due to how they work may span multiple physical volumes since an LV consumes space from the VG which can allocate anywhere in the PV's available. This can be visualised as:</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>┌──────┐ ┌──────────────────────┐ </span><span>│ LV │ │ LV │ </span><span>└──────┘ └──────────────────────┘ </span><span>┌──────────────────────────────────────────┐ </span><span>│ VG │ </span><span>└──────────────────────────────────────────┘ </span><span>┌────────────┐ ┌────────────┐ ┌────────────┐ </span><span>│ PV │ │ PV │ │ PV │ </span><span>└────────────┘ └────────────┘ └────────────┘ </span></code></pre> <p>Here we have two LV's within a VG. The VG is made from 3 PVs. The storage of the LV's may be anywhere within the pool of PV's that exist. This allows PVs to be added or removed dynamically as the LV content can be moved at any time.</p> <h3 id="general-lvm-administration">General LVM Administration</h3> <h4 id="read-the-man-pages">Read The Man Pages</h4> <p>LVM has some of the <em>best man pages</em> ever put onto a Linux system. They are worth reading to understand the options you have for commands!</p> <blockquote> <p>HINT: Some flavours of OpenSUSE may not have man pages. To fix this:</p> </blockquote> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span># $EDITOR /etc/zypp/zypp.conf </span><span>... </span><span>rpm.install.excludedocs = no </span></code></pre> <p>Ensure man is installed, and reinstall lvm2 to make sure it's man pages are present.</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span># zypper install -f man lvm2 </span></code></pre> <h4 id="opensuse-install-lvm-tools">OpenSUSE - Install LVM Tools</h4> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span># zypper in lvm2 </span><span># reboot </span></code></pre> <blockquote> <p>NOTE: In some cases you may need to install kernel-default so that dm-raid's kernel module exists. This can be because kernel-default-base may lack the module.</p> </blockquote> <h4 id="show-all-physical-volumes">Show all Physical Volumes</h4> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span># pvs </span><span> PV VG Fmt Attr PSize PFree </span><span> /dev/vdb vg00 lvm2 a-- 50.00g 50.00g </span><span> /dev/vdc vg00 lvm2 a-- 50.00g 50.00g </span><span> /dev/vdd vg00 lvm2 a-- 50.00g 50.00g </span><span> /dev/vde vg00 lvm2 a-- 50.00g 50.00g </span></code></pre> <h4 id="show-all-volume-groups">Show all Volume Groups</h4> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span># vgs </span><span> VG #PV #LV #SN Attr VSize VFree </span><span> vg00 4 0 0 wz--n- 199.98g 199.98g </span></code></pre> <h4 id="show-all-logical-volumes">Show all Logical Volumes</h4> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span># lvs </span><span> LV VG Attr LSize Pool Origin Data% Meta% Move Log Cpy%Sync Convert </span><span> lv_raid vg00 rwi-a-r--- 80.00g </span></code></pre> <p>Show the internal details of LVs</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span># lvs -a </span><span> LV VG Attr LSize Pool Origin Data% Meta% Move Log Cpy%Sync Convert </span><span> lv_raid vg00 rwi-a-r--- 80.00g 18.80 </span><span> [lv_raid_rimage_0] vg00 Iwi-aor--- 40.00g </span><span> [lv_raid_rimage_1] vg00 Iwi-aor--- 40.00g </span><span> [lv_raid_rimage_2] vg00 Iwi-aor--- 40.00g </span><span> [lv_raid_rmeta_0] vg00 ewi-aor--- 4.00m </span><span> [lv_raid_rmeta_1] vg00 ewi-aor--- 4.00m </span><span> [lv_raid_rmeta_2] vg00 ewi-aor--- 4.00m </span></code></pre> <p>Show the internal details of LVs and which devices are backing their storage.</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span># lvs -a -o +devices </span><span> LV VG Attr LSize Pool Origin Data% Meta% Move Log Cpy%Sync Convert Devices </span><span> lv_raid vg00 rwi-a-r--- 80.00g 21.35 lv_raid_rimage_0(0),lv_raid_rimage_1(0),lv_raid_rimage_2(0) </span><span> [lv_raid_rimage_0] vg00 Iwi-aor--- 40.00g /dev/vdb(1) </span><span> [lv_raid_rimage_1] vg00 Iwi-aor--- 40.00g /dev/vdc(1) </span><span> [lv_raid_rimage_2] vg00 Iwi-aor--- 40.00g /dev/vdd(1) </span><span> [lv_raid_rmeta_0] vg00 ewi-aor--- 4.00m /dev/vdb(0) </span><span> [lv_raid_rmeta_1] vg00 ewi-aor--- 4.00m /dev/vdc(0) </span><span> [lv_raid_rmeta_2] vg00 ewi-aor--- 4.00m /dev/vdd(0) </span></code></pre> <h3 id="using-lvm-for-raid">Using LVM For Raid</h3> <p>First you need to select your devices that will become the PV's</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span># lsblk </span><span>NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINTS </span><span>sr0 11:0 1 372K 0 rom </span><span>vda 254:0 0 10G 0 disk </span><span>├─vda1 254:1 0 2M 0 part </span><span>├─vda2 254:2 0 33M 0 part /boot/efi </span><span>└─vda3 254:3 0 10G 0 part / </span><span>vdb 254:16 0 50G 0 disk ---- </span><span>vdc 254:32 0 50G 0 disk | &lt;-- I will use these 4 devices. </span><span>vdd 254:48 0 50G 0 disk | </span><span>vde 254:64 0 50G 0 disk ---- </span></code></pre> <p>Create PVs on each member device.</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span># pvcreate /dev/vdb </span><span># pvcreate /dev/disk/by-path/virtio-pci-0000\:09\:00.0 </span><span>... </span></code></pre> <p>Create a VG containing all the PVs</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>vgcreate &lt;name of vg&gt; &lt;path to pv&gt; [&lt;path to pv&gt; ...] </span><span># vgcreate vg00 /dev/vdb /dev/vdc /dev/vdd /dev/vde </span></code></pre> <p>Create a new LV that is at your prefered raid level.</p> <ul> <li>If you have two PV's choose raid 1</li> <li>If you have three or more choose between raid 5 or raid 10</li> </ul> <p>In each case, LVM will make sure that the data of the LV is correctly split to PV's to ensure redundancy.</p> <ul> <li>Raid 1 mirrors. It has no performance changes.</li> <li>Raid 5/6 have better <em>write</em> performance but lower <em>read</em> performance compared to raid 10.</li> <li>Raid 10 has better <em>read</em> performance but lower <em>write</em> performance compared to raid 5/6</li> </ul> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>lvcreate [options] -n &lt;name of LV&gt; --type &lt;type&gt; [-L|-l] &lt;size of lv&gt; --raidintegrity y &lt;VG to create the LV in&gt; </span><span>## Create an lv that consumes all space in the VG </span><span>## NOTE: you may need to reduce this from 100% with raidintegrity to allow LVM the space </span><span>## to create the LV. </span><span># lvcreate -n lv_raid --type raid10 -l 100%FREE --raidintegrity y vg00 </span><span>## Create an lv that provides 80G of raid storage, but consumers more of the VG underneath </span><span># lvcreate -n lv_raid --type raid5 -L 80G --raidintegrity y vg00 </span></code></pre> <blockquote> <p>HINT: The raidintegrity flag enables checksums of extents to allow detection of potential disk corruption</p> </blockquote> <p>You can then use <code>lvs</code> to show the state of the sync process of the LV.</p> <p>Once created you can make a new filesystem on the volume.</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>mkfs.&lt;fs name&gt; /dev/&lt;vg name&gt;/&lt;lv name&gt; </span><span># mkfs.xfs /dev/vg00/lv_raid </span></code></pre> <p>This can then be added to <code>/etc/fstab</code> to mount on boot.</p> <blockquote> <p>HINT: <code>/dev/&lt;vg name&gt;/&lt;lv name&gt;</code> paths will never change and can be used reliably in fstab</p> </blockquote> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span># $EDITOR /etc/fstab </span><span>... </span><span>/dev/vg00/lv_raid /mnt/raid xfs defaults 0 0 </span></code></pre> <h3 id="managing-lvm-raid">Managing LVM Raid</h3> <h4 id="replacing-a-working-disk">Replacing a Working Disk</h4> <p>If you want to expand your array, or just replace an old piece of disk media you can do this live.</p> <p>Attach the new disk and locate it with <code>lsblk</code>. Note it's name, path or other identifier.</p> <p>Locate the disk you want to replace by it's path or identifier.</p> <p>Create a new PV on the new disk.</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span># pvcreate /dev/vdf </span></code></pre> <p>Extend the VG with the new PV</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span># vgextend vg00 /dev/vdf </span></code></pre> <p>Check the pv was added to the vg</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span># pvs </span></code></pre> <p>Move the extents from the original device, to the new device.</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>## This blocks and monitors the move. If you ctrl-c it continues in the background. </span><span># pvmove /dev/vde /dev/vdf </span><span>## Run the move in the background </span><span># pvmove -b /dev/vde /dev/vdf </span></code></pre> <p>If backgrounded, you can monitor the move with:</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span># lvs -a </span></code></pre> <h4 id="replacing-a-corrupted-missing-disk">Replacing a Corrupted/Missing Disk</h4> <p>When checking lvs if a disk is corrupted or missing you will see errors such as:</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span># lvs </span><span> WARNING: Couldn&#39;t find device with uuid weDidc-rBve-EL25-NqTG-X2n6-fGmW-FGCs9l. </span><span> WARNING: VG vg00 is missing PV weDidc-rBve-EL25-NqTG-X2n6-fGmW-FGCs9l (last written to /dev/vdd). </span><span> WARNING: Couldn&#39;t find all devices for LV vg00/lv_raid_rimage_2 while checking used and assumed devices. </span><span> WARNING: Couldn&#39;t find all devices for LV vg00/lv_raid_rmeta_2 while checking used and assumed devices. </span><span> LV VG Attr LSize Pool Origin Data% Meta% Move Log Cpy%Sync Convert </span><span> lv_raid vg00 rwi-a-r-p- 99.98g 100.00 </span></code></pre> <p>Create a new PV on a replacement disk, and add it to the vg.</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span># pvcreate /dev/path/to/disk </span><span># vgextend vg00 /dev/path/to/disk </span></code></pre> <p>The logical volume must be active to initiate the replacement:</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span># lvchange -ay vg00/lv_raid </span></code></pre> <p>Replace the failed device allocating the needed extents.</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>## To any free device in the VG </span><span># lvconvert --repair vg00/lv_raid </span><span>## To a specific PV </span><span># lvconvert --repair vg00/lv_raid /dev/vde </span></code></pre> <p>You can view the progress of the repair with <code>lvs</code></p> <p>Instruct the vg to remove the missing device metadata now that the replacement is complete.</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span># vgreduce --removemissing vg00 </span></code></pre> Starting with Rage on OpenSUSE Fri, 26 May 2023 00:00:00 +0000 Unknown https://fy.blackhats.net.au/blog/2023-05-26-starting-with-rage-on-opensuse/ https://fy.blackhats.net.au/blog/2023-05-26-starting-with-rage-on-opensuse/ <h1 id="starting-with-rage-on-opensuse">Starting with Rage on OpenSUSE</h1> <p>Rage is a rust implementation of Age, a modern, simple and secure file encryption tool. It is easier to use than other tools like GPG, and being written in a memory safe language it avoids many of the exploits that may occur in C based tools.</p> <h2 id="installing-rage">Installing Rage</h2> <p>You can install rage on leap or tumbleweed from zypper</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>zypper install rage-encryption </span></code></pre> <p>Alternately you can install from cargo with</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>cargo install rage </span></code></pre> <h2 id="key-management">Key management</h2> <p>The recipient must generate a key. This can be either a rage key, or an ssh key which is of the form <code>ssh-rsa</code> or <code>ssh-ed25519</code>.</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span># The public key is displayed. </span><span>rage-keygen -o ~/rage.key </span><span># age1y2lc7x59jcqvrpf3ppmnj3f93ytaegfkdnl5vrdyv83l8ekcae4sexgwkg </span></code></pre> <p>To use ssh keys, you can generate a key with:</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>ssh-keygen -t ed25519 </span><span># cat /root/.ssh/id_ed25519.pub </span><span>ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIE1kWXiYIn/VWzo0DlnIp3cRx/kZd6d9WbwehKJpPx1R </span></code></pre> <h2 id="encrypting-to-a-public-key">Encrypting to a public key</h2> <p>You encrypt a file to the owner of the public key with:</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>rage -e -r &lt;pub key&gt; -o &lt;encrypted output&gt; &lt;input&gt; </span><span># With their rage public key. </span><span>rage -e -r age1y2lc7x59jcqvrpf3ppmnj3f93ytaegfkdnl5vrdyv83l8ekcae4sexgwkg -o ~/encyrpted.age data </span></code></pre> <p>Alternately, if you want to allow the content of the encrypted file to be base64 for emailing (note the -a):</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>rage -e -a -r &lt;pub key&gt; -o &lt;encrypted output&gt; &lt;input&gt; </span><span># With their rage public key. </span><span>rage -e -a -r age1y2lc7x59jcqvrpf3ppmnj3f93ytaegfkdnl5vrdyv83l8ekcae4sexgwkg \ </span><span> -o ~/encyrpted.age data </span><span> </span><span># cat /tmp/encrypted </span><span>-----BEGIN AGE ENCRYPTED FILE----- </span><span>YWdlLWVuY3J5cHRpb24ub3JnL3YxCi0+IFgyNTUxOSByaDNTNHR0dlI5RkRudmpH </span><span>NEUrT1RrQ3pZdjM5alRVYThDeG5xdTBxd1EwCjYxVGkwV05ibXlWeUN3MWVuNTBC </span><span>Qk1SdEwyd3J1RjgrNVkxem5pbHJscVEKLT4gTyI7WFQtZ3JlYXNlIEZDXyBiICFV </span><span>NgpoTlJ5ME95azMycE5GbS9oS0h6a280RlRTRHNKbE9mMGZjTmFCUjB6aWEwZGxU </span><span>Rjg1RkZmdkhBSkU4Y1dZdEM3CjV0VXl4dE5Qd3E0SU1GSXNIejQKLS0tIFhscDBn </span><span>MlBiTmxPekthY1RabVcxN0JkQnJsd3RKUkpTKzRkelZ1eDFXSk0KHzCOyBZHPe/P </span><span>cV3Fez6yusycXcm83Bt+N2yHTG2utOGxfmIxb5c= </span><span>-----END AGE ENCRYPTED FILE----- </span></code></pre> <p>The ssh public key can be encrypted to if the public key is in a file</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>rage -e -a -R &lt;path to public key&gt; -o &lt;encrypted output&gt; &lt;input&gt; </span><span># Using the ssh public key in a file </span><span>rage -e -a -R ~/id_ed25519.pub -o /tmp/ssh-encrypted data </span></code></pre> <h2 id="decrypting-a-file">Decrypting a file</h2> <p>The recipient can then decrypt with:</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>rage -d -i &lt;path to private key&gt; -o &lt;decrypted output&gt; &lt;encrypted input&gt; </span><span> </span><span>rage -d -i ~/rage.key -o /tmp/decrypted /tmp/encrypted </span><span> </span><span># cat /tmp/decrypted </span><span>hello </span></code></pre> <p>With an ssh private key</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>rage -d -i &lt;path to ssh private key&gt; -o &lt;decrypted output&gt; &lt;encrypted input&gt; </span><span> </span><span>rage -d -i ~/.ssh/id_ed25519 -o /tmp/ssh-decrypted /tmp/ssh-encrypted </span></code></pre> <h2 id="encrypt-to-multiple-recipients">Encrypt to multiple recipients</h2> <p>Rage can encrypt to multiple identities at a time.</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>rage -e -a -R &lt;first ssh pub key&gt; -R &lt;second pub key&gt; ... -o &lt;encrypted output&gt; &lt;input&gt; </span><span>rage -e -a -r &lt;first pub key&gt; -r &lt;second pub key&gt; ... -o &lt;encrypted output&gt; &lt;input&gt; </span><span>rage -e -a -R &lt;first ssh pub key&gt; -r &lt;first rage pub key&gt; ... -o &lt;encrypted output&gt; &lt;input&gt; </span><span> </span><span>rage -e -a -R /root/.ssh/id_ed25519.pub \ </span><span> -r age1h8equ4vs5pyp8ykw0z8m9n8m3psy6swme52ztth0v66frgu65ussm8gq0t \ </span><span> -r age1y2lc7x59jcqvrpf3ppmnj3f93ytaegfkdnl5vrdyv83l8ekcae4sexgwkg \ </span><span> -o /tmp/ssh-encrypted hello </span></code></pre> <p><em>all</em> of the associated keys can decrypt this file.</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>rage -d -i /root/.ssh/id_ed25519 -o /tmp/ssh-decrypted /tmp/ssh-encrypted </span><span> </span><span>rage -d -i ~/rage.key -o /tmp/decrypted /tmp/ssh-encrypted </span></code></pre> <h2 id="using-a-passphrase-instead-of-a-key">Using a passphrase instead of a key</h2> <p>Rage can encrypt with a passphrase:</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>rage -e -p -o &lt;encrypted output&gt; &lt;input&gt; </span><span> </span><span>rage -e -p -o /tmp/passphrase-encrypted data </span></code></pre> <p>Decrypted with (passphrase is detected and prompted for):</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>rage -d -o &lt;decrypted output&gt; &lt;encrypted input&gt; </span><span> </span><span>rage -d -o /tmp/decrypted /tmp/passphrase-encrypted </span></code></pre> About Sat, 22 Apr 2023 00:00:00 +0000 Unknown https://fy.blackhats.net.au/pages/about/ https://fy.blackhats.net.au/pages/about/ <h1 id="about">About</h1> <p>I'm William (Firstyear), a software engineer in Brisbane, Australia. I'm part of the <a href="http://www.port389.org">389 Directory Server</a> team, working for SUSE Labs. I'm always happy to talk LDAP and programming, and am very happy to help out with your issues and queries.</p> <p>I can be contacted about entries on my blog at william at blackhats dot net dot au. If there is anything you would like to see on this blog, I'd be happy to hear from you.</p> <p>Happy reading!</p> Using a TPM for SSH keys on OpenSUSE Tumbleweed Thu, 20 Apr 2023 00:00:00 +0000 Unknown https://fy.blackhats.net.au/blog/2023-04-20-using-a-tpm-for-ssh-keys-on-opensuse-tumbleweed/ https://fy.blackhats.net.au/blog/2023-04-20-using-a-tpm-for-ssh-keys-on-opensuse-tumbleweed/ <h1 id="using-a-tpm-for-ssh-keys-on-opensuse-tumbleweed">Using a TPM for SSH keys on OpenSUSE Tumbleweed</h1> <p>In some environments it is required to store ssh private keys in a way where they can not be extracted from the machine. Trusted Platform Modules (TPM) are an excellent way to achieve this. While other guides exist online for how to configure this for other distributions, this will focus on OpenSUSE Tumbleweed.</p> <h2 id="install-packages">Install Packages</h2> <p>The following is required to be installed.</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>zypper install tpm2-pkcs11 tpm2.0-tools tpm2-0-tss libtpm2_pkcs11-0 </span></code></pre> <ul> <li><code>tpm2-pkcs11</code> - tools to configure keys in a tpm via PKCS11</li> <li><code>tpm2.0-tools</code> - tools for TPM introspection</li> <li><code>tpm2-0-tss</code> - udev rules and tss group</li> <li><code>libtpm2_pkcs11-0</code> - library for ssh to access TPM via PKCS11</li> </ul> <h2 id="check-the-tpm-exists">Check the TPM exists</h2> <p>You can check the TPM exists and is working with:</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>ls -l /dev/tpm* </span><span># crw-rw---- 1 tss root 10, 224 Apr 19 18:39 /dev/tpm0 </span></code></pre> <p>To check the supported algorithms on the TPM:</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>tpm2_getcap algorithms </span></code></pre> <p>If this command errors, your TPM may be misconfigured or you may not have access to the TPM.</p> <p><em>HINT</em>: You can add a TPM to a KVM virtual machine with virt-install with the line:</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>--tpm backend.type=emulator,backend.version=2.0,model=tpm-tis </span></code></pre> <p>From virt-manager you can add the TPM via &quot;Add Hardware&quot;, &quot;TPM&quot;.</p> <p>Editing the virtual machine xml directly a TPM can be defined with:</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>&lt;domain type=&#39;kvm&#39;&gt; </span><span> &lt;devices&gt; </span><span> &lt;tpm model=&#39;tpm-tis&#39;&gt; </span><span> &lt;backend type=&#39;emulator&#39; version=&#39;2.0&#39;/&gt; </span><span> &lt;/tpm&gt; </span><span> &lt;/devices&gt; </span><span>&lt;/domain&gt; </span></code></pre> <h2 id="allow-user-access">Allow User Access</h2> <p>Add your user to the tss group</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>usermod -a -G tss username </span></code></pre> <h2 id="configure-the-ssh-key">Configure the SSH key</h2> <p><em>NOTE</em> Be sure to perform these steps as your user - not as root!</p> <p>Initialise the tpm PKCS11 store - note the id in the output.</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>tpm2_ptool init </span><span># action: Created </span><span># id: 1 </span></code></pre> <p>Using the id from the above output, you can use that to create a new token. Note here that the userpin is the pin for using the ssh key. The sopin is the recovery password incase you lose the pin and need to reset it.</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>tpm2_ptool addtoken --pid=1 --label=ssh --userpin=&quot;&quot; --sopin=&quot;&quot; </span><span>tpm2_ptool addkey --label=ssh --userpin=&quot;&quot; --algorithm=ecc256 </span></code></pre> <p>If you want to use a different key algorithm, other choices are rsa2048, rsa3072, rsa4096, ecc256, ecc384, ecc521.</p> <p>Now you can view the public key with:</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>ssh-keygen -D /usr/lib64/pkcs11/libtpm2_pkcs11.so.0 </span></code></pre> <p>It's a good idea to store this into a file for later:</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>ssh-keygen -D /usr/lib64/pkcs11/libtpm2_pkcs11.so.0 | tee ~/.ssh/id_ecdsa_tpm.pub </span></code></pre> <h2 id="using-the-ssh-key">Using the SSH Key</h2> <p>You can modify your ssh configuration to always use the key. You will be prompted for the userpin to access the ssh key as required.</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span># ~/.ssh/config </span><span> </span><span>PKCS11Provider /usr/lib64/pkcs11/libtpm2_pkcs11.so.0 </span><span>PasswordAuthentication no </span></code></pre> How Hype Will Turn Your Security Key Into Junk Thu, 02 Feb 2023 00:00:00 +0000 Unknown https://fy.blackhats.net.au/blog/2023-02-02-how-hype-will-turn-your-security-key-into-junk/ https://fy.blackhats.net.au/blog/2023-02-02-how-hype-will-turn-your-security-key-into-junk/ <h1 id="how-hype-will-turn-your-security-key-into-junk">How Hype Will Turn Your Security Key Into Junk</h1> <p>In the last few months there has been a lot of hype about &quot;passkeys&quot; and how they are going to change authentication forever. But that hype will come at a cost.</p> <p>Obsession with passkeys are about to turn your security keys (yubikeys, feitian, nitrokeys, ...) into obsolete and useless junk.</p> <p>It all comes down to one thing - resident keys.</p> <h2 id="what-is-a-resident-key">What is a Resident Key</h2> <p>To understand the problem, we need to understand what a discoverable/resident key is.</p> <p>You have probably seen that most keys support an 'unlimited' number of accounts. This is achieved by sending a &quot;key wrapped key&quot; to the security key. When the Relying Party (Authentication Server) wants to authenticate your security key, it will provide you a &quot;credential id&quot;. That credential ID is an encrypted blob that only your security key can decrypt. If your security key can decrypt that blob it yields a private key that is specific to that single RP that you can use for signatures.</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>┌────────────────┐ ┌────────────────┐ ┌────────────────┐ </span><span>│ Relying Party │ │ │ Browser │ │ │ Security Key │ </span><span>└────────────────┘ └────────────────┘ └────────────────┘ </span><span> │ │ </span><span> 1. Send </span><span> Credential ──────────┼───────▶ │ </span><span> IDs </span><span> │ 2. Forward to─────────┼─────▶ 3. Decrypt </span><span> Security Key Credential ID </span><span> │ │ with Master Key </span><span> │ </span><span> │ │ │ </span><span> │ </span><span> │ │ ▼ </span><span> 4. Sign Challenge </span><span> │ │ with Decrypted </span><span> Key </span><span> │ │ │ </span><span> │ </span><span> │ │ │ </span><span> ▼ </span><span> │ 6. Return ◀──────┼───────── 5. Return </span><span> ◀──────────────── Signature Signature </span><span> │ │ </span></code></pre> <p>This is what is called a non-resident or non-discoverable credential. The reason is that the private key can <em>not be discovered</em> by the security key without the Credential ID provided to it externally. This is because the private keys are <em>not resident</em> inside the security enclave - only the master key is.</p> <p>Contrast to this, a resident key or discoverable credential is one where the private key is <em>stored</em> in the security key itself. This allows the security key to discover (hence the name) what private keys might be used in the authentication.</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>┌────────────────┐ ┌────────────────┐ ┌────────────────┐ </span><span>│ Relying Party │ │ │ Browser │ │ │ Security Key │ </span><span>└────────────────┘ └────────────────┘ └────────────────┘ </span><span> │ │ </span><span> 1. Send Empty </span><span> CredID list──────────┼───────▶ │ </span><span> 2. Query ───────────────▶ </span><span> │ Security Key │ 3. Discover Keys </span><span> for RP </span><span> │ 4. Select a ◀────────┼───── </span><span> Security Key </span><span> │ ─────────┼────▶ 5. Sign Challenge </span><span> with Resident Key </span><span> │ │ │ </span><span> │ </span><span> │ │ │ </span><span> ▼ </span><span> │ 7. Return ◀───────┼──────── 6. Return </span><span> ◀──────────────── Signature Signature </span><span> │ │ </span><span> </span><span> │ │ </span><span> </span><span> │ │ </span></code></pre> <p>Now, the primary difference here is that resident/discoverable keys consume <em>space</em> on the security key to store them since they need to persist - there is no credential id to rely on to decrypt with our master key!</p> <h3 id="are-non-resident-keys-less-secure">Are non-resident keys less secure?</h3> <p>A frequent question here is if non resident keys are less secure than resident ones. Credential ID's as key wrapped keys are secure since they are encrypted with aes128 and hmaced. This prevents them being tampered with or decrypted by an external source. If aes128 were broken and someone could decrypt your private-key in abscence of your security key, they probably could also break TLS encyrption, attack ssh and do much worse. Your key wrapped keys rely on the same security features that TLS relies on.</p> <h3 id="how-does-userverification-factor-in-here">How does userverification factor in here?</h3> <p>Another frequent question (or confusion) is that a credential needs to be resident to enforce userVerification. That's not the case! Your device can assert not just presence by touch, but that it's really you holding the device by internally validating a PIN or biometric. This is governed by the userVerification flags which are seperate to residency. This allows your device to be &quot;a self contained multifactor authenticator&quot;. Very cool!</p> <h2 id="resident-keys-and-your-security-key">Resident Keys and Your Security Key</h2> <p>Now that we know what a resident key is, we can look at how these work with your security keys.</p> <p>Since resident keys store their key in the device, this needs to consume <em>space</em> inside the security key. Generally, resident key slots are at a premium on a security key. Nitrokeys for example only support 8 resident keys. Yubikeys generally support between 20 and 32. Some keys support <em>no</em> resident keys at all.</p> <p>The other problem is what CTAP standard your key implements. There are three versions - CTAP2.0, CTAP2.1PRE and CTAP2.1.</p> <p>In CTAP2.1 (the latest) and CTAP2.1PRE you can individually manage, update and delete specific resident keys from your device.</p> <p>In CTAP2.0 however, you can not. You can not delete a residentkey without <em>resetting the whole device</em>. Resetting the device also resets your master key, meaning all your non-resident keys will no longer work either. This makes resident keys on a CTAP2.0 device a serious commitment. You really don't want to accidentally fill up that limited space you have!</p> <p>In most cases, your key is <em>very likely</em> to be CTAP2.0. For example some manufacturers like yubico, they're keys CTAP version is defined by their firmware version. I have multiple yk5 series keys that have different levels of CTAP support.</p> <h2 id="so-why-are-resident-keys-a-problem">So Why Are Resident Keys a Problem?</h2> <p>On their own, and used carefully resident keys are great for certain applications. The problem is the hype and obsession with <em>passkeys</em>.</p> <p>In 2022 Apple annouced their passkeys feature on MacOS/iOS allowing the use of touchid/faceid as a webauthn authenticator similar to your security key. Probably quite wisely, rather than calling them &quot;touchid&quot; or &quot;credentials&quot; or &quot;authenticators&quot; Apple chose to have a nicer name for users. Honestly passkeys is a good name rather than &quot;webauthn authenticator&quot; or &quot;security key&quot;. It evokes a similar concept to passwords which people are highly accustomed to, while also being different enough with the 'key' to indicate that it operates in a different way.</p> <p>The problem (from an external view) is that passkeys was a branding or naming term of something - but overnight authentication thought leaders needed to be <em>on the hype</em>. &quot;What is a passkey?&quot;. Since Apple didn't actually define it, this left a void for our thought leaders to answer that question for users hungry to know &quot;what indeed is a passkey?&quot;.</p> <p>As a creator of a relying party and the webauthn library for Rust, we defined passkeys as the name for &quot;all possible authenticators&quot; that a person may choose to use. We wanted to support the goal to remove and eliminate passwords, and passkeys are a nice name for this.</p> <p>Soon after that, some community members took to referring to passkeys to mean &quot;credentials that are synchronised between multiple devices&quot;. This definition is at the least, not harmful, even if it doesn't express that there are many possible types of authenticators that can be used.</p> <p>Some months later a person took the stage at FIDO's Authenticate conference and annouced &quot;a passkey is a resident key&quot;. Because of the scale and size of the platform, this definition has now stuck. This definition has become so invasive that even <em>FIDO</em> now use it as <a href="https://fidoalliance.org/passkeys/#faq">their definition</a>.</p> <p>Part of the reason this definition is hyped is because it works with an upcoming browser feature that allows autocomplete of a username and webauthn credential if the key is resident. You don't have to type your username. This now means that we have webauthn libraries pushing for residentkey as a requirement for all registrations, and many people will follow this advice without seeing the problem.</p> <p>The problem is that security keys with their finite storage and lack of credential management will fill up <em>rapidly</em>. In my password manager I have more than 150 stored passwords. If all of these were to become resident keys I would need to buy at least 5 yubikeys to store all the accounts, and then another 5-10 as &quot;backups&quot;. I really don't want to have to juggle and maintain 10 to 15 yubikeys ...</p> <p>This is an awful user experience to put it mildly. People who choose to use security keys, now won't be able to due to passkeys resident key requirements. What will also confuse users is this comes on the tail of FIDO certified keys marketing with statements (which are true with non-resident keys) like:</p> <p><em>Infinite key pair storage</em></p> <p><em>There is no limit to the number of accounts registered in [redacted] FIDO® Security Key.</em></p> <p>To add further insult, an expressed goal of the Webauthn Work Group is that users should always be free to choose any authenticator they wish without penalty. Passkeys forcing key residency flies directly in the face of this.</p> <p>This leaves few authenticator types which will work properly in this passkey world. Apples own passkeys, Android passkeys, password managers that support webauthn, Windows with TPM 2.0, and Chromium based browsers on MacOS (because of how they use the touchid as a TPM).</p> <h2 id="what-can-be-done">What Can Be Done?</h2> <h3 id="submit-to-the-webauth-wg-browsers-to-change-rk-preferred-to-exclude-security-keys">Submit to the Webauth WG / Browsers to change rk=preferred to exclude security keys</h3> <p>Rather than passkeys being resident keys, passkeys could be expanded to be all possible authenticators where some subset opportunistically are resident. This puts passwordless front and center with residency as a bonus ui/ux for those who opt to use devices that support unlimited resident keys.</p> <p>Currently there are three levels of request an RP can make to request resident keys. Discouraged, Preferred and Required. Here is what happens with different authenticator types when you submit each level.</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span> ┌────────────────────┬────────────────────┬────────────────────┐ </span><span> │ Roaming │ Platform │ Platform │ </span><span> │ Authenticator │ Authenticator │ Authenticator │ </span><span> │ (Yubikey) │(Android Behaviour) │ (iOS Behaviour) │ </span><span> └────────────────────┴────────────────────┴────────────────────┘ </span><span> ┌────────────────────┐ ┌────────────────────┬────────────────────┬────────────────────┐ </span><span> │ │ │ │ │ │ </span><span> │ rk=discouraged │ │ RK false │ RK false │ RK true │ </span><span> │ │ │ │ │ │ </span><span> ├────────────────────┤ ├────────────────────┼────────────────────┼────────────────────┤ </span><span> │ │ │ │ │ │ </span><span> │ rk=preferred │ │ RK true (!) │ RK true │ RK true │ </span><span> │ │ │ │ │ │ </span><span> ├────────────────────┤ ├────────────────────┼────────────────────┼────────────────────┤ </span><span> │ │ │ │ │ │ </span><span> │ rk=required │ │ RK true │ RK true │ RK true │ </span><span> │ │ │ │ │ │ </span><span> └────────────────────┘ └────────────────────┴────────────────────┴────────────────────┘ </span></code></pre> <p>Notice that in rk=preferred the three columns behave the same as rk=required?</p> <p>Rather than passkeys setting rk=required, if rk=preferred were softened so that on preferred meant &quot;create a resident key only if storage is unlimited&quot; then we would have a situation where Android/iOS would always get resident keys, and security keys would not have space consumed.</p> <p>However, so far the WG is resistant to this change. It is not out of the question that browsers could implement this change externally, but that would in reality be down to the chrome team to decide.</p> <h3 id="insist-on-your-passkey-library-setting-rk-discouraged">Insist on your Passkey library setting rk=discouraged</h3> <p>Rather than rk=required which excludes security keys, rk=discouraged is the next best thing. Yes it means that android users won't get conditional UI. But what do we prefer - some people have to type a username (that already has provisions to autocomplete anyway). Or do we damage and exclude security keys completely?</p> <h3 id="contact-fido-and-request-rk-storage-as-a-certification-feature">Contact FIDO and request RK storage as a certification feature</h3> <p>Currently FIDO doesn't mandate any amount of storage requirements for certified devices. Given that FIDO also seem to want resident keys, then they should also mandate that certified devices have the ability to store thousands of resident keys. This way as a consumer you can pick and select certified devices.</p> <h3 id="something-else">Something Else?</h3> <p>If you have other ideas on how to improve this let me know!</p> <h2 id="conclusion">Conclusion</h2> <p>The hype around passkeys being resident keys will prevent - or severly hinder - users of security keys from choosing the authenticator they want to use online in the future.</p> Why are PBKDF2-SHA256 and PBKDF2_SHA256 different in 389-ds? Fri, 25 Nov 2022 00:00:00 +0000 Unknown https://fy.blackhats.net.au/blog/2022-11-25-why-are-pbkdf2-sha256-and-pbkdf2-sha256-different-in-389-ds/ https://fy.blackhats.net.au/blog/2022-11-25-why-are-pbkdf2-sha256-and-pbkdf2-sha256-different-in-389-ds/ <h1 id="why-are-pbkdf2-sha256-and-pbkdf2-sha256-different-in-389-ds">Why are PBKDF2-SHA256 and PBKDF2_SHA256 different in 389-ds?</h1> <p>In a mailing list discussion recently it came up about what password hash format should you use in 389-ds. Confusingly we have two PBKDF2 SHA256 implementations, which has a bit of history.</p> <h2 id="too-lazy-didn-t-read">Too Lazy, Didn't Read</h2> <p>Use PBKDF2-SHA256. (hyphen, not underscore).</p> <h2 id="what-s-pbkdf2-anyway">What's PBKDF2 anyway?</h2> <p>Passwords are a shared-knowledge secret, so knowledge of the password allows you to authenticate as the person. When we store that secret, we don't want it stored in a form where a person can steal and use it. This is why we don't store passwords cleartext - A rogue admin or a database breach would leak your passwords (and people do love to re-use their passwords over many websites ...)</p> <p>Because of this authentication experts recommend <em>hashing</em> your password. A one-way hash function given an input, will always produce the same output, but given the hash output, you can not derive the input.</p> <p>However, this also isn't the full story. Simply hashing your password isn't enough because people have found many other attacks. These include things like rainbow tables which are a compressed and precomputed &quot;lookup&quot; of hash outputs to their inputs. You can also bruteforce dictionaries of common passwords to see if they match. All of these processes for an attacker use their CPU to generate these tables or bruteforce the passwords.</p> <p>Most hashes though are designed to be <em>fast</em> and in many cases your CPU has hardware to accelerate and speed these up. All this does is mean that if you use a <em>verification hash</em> for storing passwords then an attacker just can attack your stored passwords even faster.</p> <p>To combat this, what authentication experts truly recommend is <em>key derivation functions</em>. A key derivation function is similar to a hash where an input always yields the same output, but a KDF also intends to be resource consuming. This can be ram or cpu time for example. The intent is that an attacker bruteforcing your KDF hashed passwords should have to expend a large amount of CPU time and resources, while also producing far fewer results.</p> <h2 id="how-is-this-related-to-vs">How is this related to - vs _?</h2> <p>In 389-ds when I implemented PBKDF2_SHA256 I specifically chose to use the '_' (underscore) to not conflict to '-' which is the OpenLDAP PBKDF2 scheme. We have differences in how we store our PBKDF2_SHA256 in the userPassword value so they aren't compatible. At the time since I was writing the module in C it was easier to use our own internal format than to attempt C String manipulation which is a common source of vulnerabilities. I opted to use a binary array with fixed lengths and offsets so that we could do bound checks rather than attempting to split a string with delimiters (and yes, I accounted for endianness in the design).</p> <p>The issue however is that after we implemented this we ran into a problem with NSS (the cryptographic library) that 389-ds uses. NSS PBKDF2 implementation is flawed, and is 4 times slower (I think, I can't find the original report from RedHat Bugzilla) than OpenSSL for the same result. This means that for 389-ds to compute a password hash in say ... 1 second, OpenSSL can do the same in 0.25 seconds. Since we have response time objectives we wish to meet, this forced 389-ds to use fewer PBKDF2 rounds, well below the NIST SP800-63b recommendations.</p> <p>For a long time we accepted this because it was still a significant improvement over our previous salted sha256 single round implementation, and we hoped the NSS developers would resolve the issue. However, they did not fix it and did not accept it was a security issue.</p> <p>Since then, we've added support for Rust into 389-ds and had a growing interest to migrate from OpenLDAP to 389-ds. To support this, I added support for OpenLDAP formatted password hashes to be directly imported to 389-ds with a Rust plugin called pwdchan that is now part of 389-ds by default. In this plugin we used the OpenSSL cryptographic provider which does not have the same limitations as NSS, meaning we can increase the number of PBKDF2 rounds to the NIST SP800-63b recommendations without sacrificing large amounts of (wasted) CPU time.</p> <h2 id="conclusion">Conclusion</h2> <p>Use PBKDF2-SHA256.</p> <ul> <li>It's written in Rust.</li> <li>It meets NIST SP800-63b recommendations.</li> </ul> Why Decentralised ID Won't Work Thu, 17 Nov 2022 00:00:00 +0000 Unknown https://fy.blackhats.net.au/blog/2022-11-17-why-decentralised-id-won-t-work/ https://fy.blackhats.net.au/blog/2022-11-17-why-decentralised-id-won-t-work/ <h1 id="why-decentralised-id-won-t-work">Why Decentralised ID Won't Work</h1> <p>Thanks to a number of <a href="https://www.abc.net.au/news/2022-09-27/optus-data-breach-cyber-attack-hacker-ransom-sorry/101476316">high profile</a> and damaging <a href="https://www.abc.net.au/news/2022-11-09/medibank-data-release-dark-web-hackers/101632088">security incidents</a> in Australia people have once again been discussing Decentralised ID (DID). As someone who has spent most of the career working on identity management, I'm here to tell you why <em>it will not work</em>.</p> <h2 id="what-is-decentralised-id-trying-to-do">What Is Decentralised ID Trying To Do?</h2> <p>To understand what DID is trying to achieve we have to look at what a &quot;centralised&quot; system is doing.</p> <p>Lets consider an account holder like Google. You create an account with them, and you store your name and some personal data, as well as a method of authentication, such as a password and OTP, or Webauthn.</p> <p>Now you go to some other website and it says &quot;login with Google&quot;. That site redirects to Google, who authenticates you, and then the website trusts Google to say &quot;yes or no&quot; that &quot;you are who you say you are&quot;. You can consent to this website seeing details about you like an email address or name.</p> <p>A decentralised system works differently. You present a signed metadata statement about yourself to the website, and that cryptograhic signature can be traced back to your signing private key. This cryptograhic proof attests that you are the profile/account holder.</p> <h2 id="what-does-did-claim-to-achieve">What Does DID Claim To Achieve?</h2> <ul> <li>That you are the only authority who can modify your own identity and data.</li> <li>You control who can access (view) that data.</li> <li>Cryptographic verification that an identity is who they claim to be.</li> </ul> <h2 id="this-will-never-work">This Will Never Work</h2> <h3 id="no-consideration-of-human-behaviour">No Consideration Of Human Behaviour</h3> <p>DID systems do not consider human behaviour in their design.</p> <p>I can not put it better, than <a href="https://web.archive.org/web/20221001000000*/https://bradleymonk.com/w/images/9/91/The_truth_about_Unix_Don_Norman.pdf">Don Norman, in his paper &quot;The Truth about Unix&quot;.</a></p> <p><em>System designers take note. Design the system for the person, not for the computer, not even for yourself. People are also information processing systems, with varying degrees of knowledge, varying degrees of experience. Friendly systems treat users as intelligent adults who, like normal adults, are forgetful, distracted, thinking of other things, and not quite as knowledgeable about the world as they themselves would like to be.</em></p> <p>People are not &quot;stupid&quot;. They are distracted and busy. This means they will lose their keys. They will be affected by <a href="https://www.abc.net.au/news/2022-02-28/qld-flood-brisbane-residents-assess-damage/100869034">events out of their control</a>.</p> <p>In a centralised system there are ways to recover your account when you lose your password/keys. There are systems to verify you, and help you restore from damage.</p> <p>In a DID system, if you lose your key, you lose everything. There is no recovery process.</p> <h3 id="gpg-already-failed">GPG Already Failed</h3> <p>DID is effectively a modern rehash of GPG - including it's problems. <a href="https://latacora.micro.blog/2019/07/16/the-pgp-problem.html">Many others have</a> lamented <a href="https://words.filippo.io/giving-up-on-long-term-pgp/">at length about</a>. These people have spent their lives studying cryptograhpic systems, and they have given up on it. Pretty much every issue they report here, applies to DID and all it's topics.</p> <h3 id="long-term-keys">Long Term Keys</h3> <p>One of the biggest issues in DID is that the identity is rooted in a private key that is held by an individual. This encourages long-term keys, which have a large blast radius (complete take over of your identity). This causes dramatic failure modes. To further this, it also prevents improvement of the cryptograhic quality of the key. When I started in IT RSA 1024 bit was secure. Now it's not. Keys need to be short lived and disposable.</p> <h3 id="you-won-t-own-your-own-data">You Won't Own Your Own Data</h3> <p>When you send a DID signed document to a provider, lets say your Bank to open a new account, what do you think they will do with that data?</p> <p>They won't destroy it and ask you for it every time you need it. They will store a copy on their servers for their records. There are often <em>extremely good reasons</em> they need to store that data as well.</p> <p>Which means that your signed document of data is performative, and the data will just be used and extracted as usual.</p> <p>DID does not solve the problem of data locality or retention. Regulation and oversight does.</p> <h3 id="trust-is-a-social-problem">Trust Is A Social Problem</h3> <p>You can't solve social problems with technology.</p> <p>The whole point of DID is about solving <em>trust</em>. Who do you <em>trust</em> to store (modify) or view your personal information?</p> <p>In a DID world, you need to be &quot;your own personal central data authority&quot; (because apparently you can't trust anyone else). That means you need to store your data, protect it from destruction and secure it from compromise.</p> <p>In the current world, for all of Google's and many other companies flaws, they still have dedicated security teams, specialists in risk analysis, and people who have dedicated themselves to protecting your accounts and your data.</p> <p>The problem is that most software engineers fall into the fallacy that because they are an expert in their subject matter, they are now an expert on identity and computer security. They are likely not security experts (the same people love to mansplain authentication to me frequently, and generally this only serves to inform me that they actually do not understand authentication).</p> <p>Why should anyone trust your DID account, when you likely have no data hygiene and insecure key storage? Why should a Bank? Why should Google? Your workplace? No one has a reason to trust you and your signatures.</p> <p>Yes there are problems with centralised identity systems - but DID does not address them, and actually may make them significantly worse.</p> <h3 id="your-verification-mark-means-nothing">Your Verification Mark Means Nothing</h3> <p>Some DID sites claim things like &quot;being able to prove ownership of an account&quot;.</p> <p>How does this proof work? Can people outside of the DID community explain these proofs? Can your accountant? Your Taxi driver?</p> <p>What this will boil down to a green tick that people will trust. It doesn't take a lot of expertise to realise that the source code for this tick can be faked pretty easily since it's simply a boolean check.</p> <p>These verification marks come back to &quot;trust&quot;, which DID does not solve. You need to <em>trust</em> the site you are viewing to render things in a certain way, the same way you have to <em>trust</em> them not to go and impersonate you.</p> <p>Even if you made a DID private key with ED25519 and signed some toots, Mastodon instance owners could still impersonate you if they wanted.</p> <p>And to further this, how is the average person expected to verify your signatures? HTTPS has already shown that the majority of the public does not have the specific indepth knowledge to assess the legitimacy of a certificate authority. How are we expecting people to now verify every other person as their own CA?</p> <p>The concept of web of trust is a performative act.</p> <p>Even <a href="https://xkcd.com/1181/">XKCD nailed this</a>.</p> <h2 id="conclusion">Conclusion</h2> <p>DID won't work.</p> <p>There are certainly issues with central authorities, and DID solves none of them.</p> <p>It is similar to <a href="/blog/html/2021/05/12/compiler_bootstrapping_can_we_trust_rust.html">bootstraping compilers</a>. It is a problem that is easy to articulate, emotionally catchy, requires widespread boring social solutions, but tech bros try to solve it unendingly with hyper-complex-technical solutions that won't work.</p> <p>You're better off just adding FIDO2 keys to your accounts and moving on.</p> Where to start with linux authentication? Wed, 24 Aug 2022 00:00:00 +0000 Unknown https://fy.blackhats.net.au/blog/2022-08-24-where-to-start-with-linux-authentication/ https://fy.blackhats.net.au/blog/2022-08-24-where-to-start-with-linux-authentication/ <h1 id="where-to-start-with-linux-authentication">Where to start with linux authentication?</h1> <p>Recently I was asked about where someone could learn how linux authentication works as a &quot;big picture&quot; and how all the parts communicate. There aren't too many great resources on this sadly, so I've decided to write this up.</p> <h2 id="who-are-you">Who ... are you?</h2> <p>The first component in linux identity is NSS or nsswitch (not to be confused with NSS the cryptography library ... ). nsswitch (name service switch) is exposed by glibc as a method to resolve uid/gid numbers and names and to then access details of the account. nsswitch can have &quot;modules&quot; that are stacked, where the first module with an answer, provides the response.</p> <p>An example of nsswitch.conf is:</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>passwd: compat sss </span><span>group: compat sss </span><span>shadow: compat sss </span><span> </span><span>hosts: files mdns dns </span><span>networks: files dns </span><span> </span><span>services: files usrfiles </span><span>protocols: files usrfiles </span><span>rpc: files usrfiles </span><span>ethers: files </span><span>netmasks: files </span><span>netgroup: files nis </span><span>publickey: files </span><span> </span><span>bootparams: files </span><span>automount: files nis </span><span>aliases: files </span></code></pre> <p>This is of the format &quot;service: module module ...&quot;. An example here is when a program does &quot;gethostbyname&quot; (a dns lookup) it accesses the &quot;host&quot; service, then resolves via files (/etc/hosts) then mdns (aka avahi, bonjour), and then dns.</p> <p>The three lines that matter for identities though, are passwd, group, and shadow. Most commonly you will use the [files]{.title-ref} module which uses [/etc/passwd]{.title-ref} and [/etc/shadow]{.title-ref} to satisfy requests. The [compat]{.title-ref} module is identical but with some extra syntaxes allowed for NIS compatibility. Another common module in nsswitch is [sss]{.title-ref} which accesses System Services Security Daemon (SSSD). For my own IDM projects we use the [kanidm]{.title-ref} nsswitch module.</p> <p>You can test these with calls to [getent]{.title-ref} to see how nsswitch is resolving some identity, for example:</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span># getent passwd william </span><span>william:x:654401105:654401105:William:/home/william:/bin/zsh </span><span># getent passwd 654401105 </span><span>william:x:654401105:654401105:William:/home/william:/bin/zsh </span><span> </span><span># getent group william </span><span>william:x:654401105:william </span><span># getent group 654401105 </span><span>william:x:654401105:william </span></code></pre> <p>Notice that both the uid (name) and uidnumber work to resolve the identity.</p> <p>These modules are dynamic libraries, and you can find them with:</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span># ls -al /usr/lib[64]/libnss_* </span></code></pre> <p>When a process wishes to resole something with nsswitch, the calling process (for example apache) calls to glibc which then loads these dylibs at runtime, and they are executed and called. This is often why the addition of new nsswitch modules in a distro is guarded and audited because these modules can end up in <em>every</em> processes memory space! This also has impacts on security as every module, and by inheritence every process, may need access [/etc/passwd]{.title-ref} or the network to do resolution of identities. Some modules improve this situation like sss, and we will give that it's own section of this blog.</p> <h2 id="prove-yourself">Prove yourself!</h2> <p>If nsswitch answers &quot;who are you&quot;, then pam (pluggable authentication modules) is &quot;prove yourself&quot;. It's what actually checks if your credentials are valid and can login or not. Pam works by having &quot;services&quot; that contact (you guessed it) modules. Most linux distros have a folder (/etc/pam.d/) which contains all the service definitions (there is a subtely different syntax in /etc/pam.conf which is not often used in linux). So lets consider when you ssh to a machine. ssh contacts pam and says &quot;I am the ssh service, can you please authorise this identity for me&quot;.</p> <p>Because this is the &quot;ssh service&quot; pam will open the named config, /etc/pam.d/SERVICE_NAME, in this case /etc/pam.d/ssh. This example is taken from Fedora, because Fedora and RHEL are very common distributions. Every distribution has their own &quot;tweaks&quot; and variants to these files, which certainly helps to make the landscape even more confusing.</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span># cat /etc/pam.d/ssh </span><span>#%PAM-1.0 </span><span>auth include system-auth </span><span>account include system-auth </span><span>password include system-auth </span><span>session optional pam_keyinit.so revoke </span><span>session required pam_limits.so </span><span>session include system-auth </span></code></pre> <p>Note the &quot;include&quot; line that is repeated four times for auth, account, password and session. These include system-auth, so lets look at that.</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span># cat /etc/pam.d/system-auth </span><span> </span><span>auth required pam_env.so </span><span>auth required pam_faildelay.so delay=2000000 </span><span>auth [default=1 ignore=ignore success=ok] pam_usertype.so isregular </span><span>auth [default=1 ignore=ignore success=ok] pam_localuser.so </span><span>auth sufficient pam_unix.so nullok </span><span>auth [default=1 ignore=ignore success=ok] pam_usertype.so isregular </span><span>auth sufficient pam_sss.so forward_pass </span><span>auth required pam_deny.so </span><span> </span><span>account required pam_unix.so </span><span>account sufficient pam_localuser.so </span><span>account sufficient pam_usertype.so issystem </span><span>account [default=bad success=ok user_unknown=ignore] pam_sss.so </span><span>account required pam_permit.so </span><span> </span><span>session optional pam_keyinit.so revoke </span><span>session required pam_limits.so </span><span>-session optional pam_systemd.so </span><span>session [success=1 default=ignore] pam_succeed_if.so service in crond quiet use_uid </span><span>session required pam_unix.so </span><span>session optional pam_sss.so </span><span> </span><span>password requisite pam_pwquality.so local_users_only </span><span>password sufficient pam_unix.so yescrypt shadow nullok use_authtok </span><span>password sufficient pam_sss.so use_authtok </span><span>password required pam_deny.so </span></code></pre> <p>So, first we are in the &quot;auth phase&quot;. This is where pam will check the auth modules for your username and password (or other forms of authentication) until a success is returned. We start at [pam_env.so]{.title-ref}, that &quot;passes but isn't finished&quot; so we go to faildelay etc. Each of these modules is consulted in turn, with the result of the module, and the &quot;rule&quot; (required, sufficient or custom) being smooshed together to create &quot;success and we are complete&quot;, &quot;success but keep going&quot;, &quot;fail but keep going&quot; or &quot;fail and we are complete&quot;. In this example, the only modules that can actually authenticate a user are [pam_unix.so]{.title-ref} and [pam_sss.so]{.title-ref}, and if neither of them provide a &quot;success and complete&quot;, then [pam_deny.so]{.title-ref} is hit which always yields a &quot;fail and complete&quot;. This phase however has only verified your <em>credentials</em>.</p> <p>The second phase is the &quot;account phase&quot; which really should be &quot;authorisation&quot;. The modules are checked once again, to determine if the module will allow or deny access to your user account to access this system. Similar rules apply where each modules result and the rules of the config combine to create a success/fail and continue/complete result.</p> <p>The third phase is the &quot;session phase&quot;. Each pam module can influence and setup things into the newly spawned session of the user. An example here is you can see [pam_limits.so]{.title-ref} which is what applies cpu/memory/filedescriptor limits to the created shell session.</p> <p>The fourth module is &quot;password&quot;. This isn't actually used in the authentication process - this stack is called when you issue the &quot;passwd&quot; command to update the users password. Each module is consulted in turn for knowledge of the account, and if they are able to alter the credentials. If this fails you will recieve a generic &quot;authentication token manipulation error&quot;, which really just means &quot;some module in the stack failed, but we wont tell you which&quot;.</p> <p>Again, these modules are all dylibs and can be found commonly in [/usr/lib64/security/]{.title-ref}. Just like nsswitch, applications that use pam are linked to [libpam.so]{.title-ref}, which inturn with load modules from [/usr/lib64/security/]{.title-ref} at runtime. Given that [/etc/shadow]{.title-ref} is root-read-only, and anything that wants to verify passwords needs to ... read this file, this generally means that any pam module is effectively running in root memory space on any system. Once again, this is why distributions carefully audit and control what packages can supply a pam module given the high level of access these require. Once again, because of how pam modules work this also generally means that the process will need network access to call out to external identity services depending on the pam modules in use.</p> <h2 id="what-about-that-network-auth">What about that network auth?</h2> <p>Now that we've covered the foundations of how processes and daemons will find details of a user and verify their credentials, lets look at SSSD which is a specific implementation of an identity resolving daemon.</p> <p>As mentioned, both nsswitch and pam have the limitation that the dylibs run in the context of the calling application, which often meant in the past with modules like [pam_ldap.so]{.title-ref} would be running in the process space of root applications, requiring network access and having to parse asn.1 (a library commonly used for remote code execution that sometimes has the side effect of encoding and decoding binary structures).</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>┌ ─ ─ ─ ─ ─ ─ ─ ─ ─ ┐ </span><span> root: uid 0 │ </span><span>│ │ </span><span> │ </span><span>│ ┌─────────────┐ │ ┌─────────────┐ </span><span> │ │ │ │ │ </span><span>│ │ │ │ │ │ </span><span> │ │ │ │ │ </span><span>│ │ SSHD │──┼────────▶│ LDAP │ </span><span> │ │ │ │ │ </span><span>│ │ │ │ │ │ </span><span> │ │ │ │ │ </span><span>│ └─────────────┘ │ └─────────────┘ </span><span> │ </span><span>│ │ Network </span><span> │ </span><span>└ ─ ─ ─ ─ ─ ─ ─ ─ ─ ┘ </span></code></pre> <p>SSSD changes this by having a daemon running locally which can be accessed by a unix socket. This allows the pam and nsswitch modules to be thin veneers with minimal functionality and surface area, who then contact an isolated daemon that does the majority of the work. This has a ton of security benefits not limited to reducing the need for the root process to decode untrusted input from the network.</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>┌ ─ ─ ─ ─ ─ ─ ─ ─ ─ ┐ ┌ ─ ─ ─ ─ ─ ─ ─ ─ ─ ┐ </span><span> root: uid 0 sssd: uid 123 │ </span><span>│ │ │ │ </span><span> │ </span><span>│ ┌─────────────┐ │ │ ┌─────────────┐ │ ┌─────────────┐ </span><span> │ │ │ │ │ │ │ </span><span>│ │ │ │ │ │ │ │ │ │ </span><span> │ │ │ │ │ │ │ </span><span>│ │ SSHD │──┼──────┼─▶│ SSSD │──┼─────────▶│ LDAP │ </span><span> │ │ │ │ │ │ │ </span><span>│ │ │ │ │ │ │ │ │ │ </span><span> │ │ │ │ │ │ │ </span><span>│ └─────────────┘ │ │ └─────────────┘ │ └─────────────┘ </span><span> │ </span><span>│ │ │ │ Network </span><span> │ </span><span>└ ─ ─ ─ ─ ─ ─ ─ ─ ─ ┘ └ ─ ─ ─ ─ ─ ─ ─ ─ ─ ┘ </span></code></pre> <p>Another major benefit of this is that SSSD can cache responses from the network in a secure way, allowing the client to resolve identities when offline. This even includes caching passwords!</p> <p>As a result this is why SSSD ends up taking on so much surface area of authentication on many distros today. With a thicc local daemon which does the more complicated tasks and work to actually identify and resolve users, and the ability to use a variety of authentication backends it is becoming widely deployed and will displace pam_ldap and pam_krb5 in the majority of network based authentication scenarioes.</p> <h2 id="inside-the-beast">Inside the beast</h2> <p>SSSD is internally built from a combination of parts that coordinate. It's useful to know how to debug these if something goes wrong:</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span># /etc/sssd/sssd.conf </span><span> </span><span>//change the log level of communication between the pam module and the sssd daemon </span><span>[pam] </span><span>debug_level = ... </span><span> </span><span>// change the log level of communication between the nsswitch module and the sssd daemon </span><span>[nss] </span><span>debug_level = ... </span><span> </span><span>// change the log level of processing the operations that relate to this authentication provider domain ``` </span><span>[domain/AD] </span><span>debug_level = ... </span></code></pre> <p>Now we've just introduced a new concept - a SSSD domain. This is different to a &quot;domain&quot; per Active Directory. A SSSD domain is just &quot;an authentication provider&quot;. A single instance of SSSD can consume identities from multiple domains at the same time. In a majority of configurations however, a single domain is configured.</p> <p>In the majority of cases if you have an issue with SSSD it is likely to be in the domain section so this is always the first place to look for debugging.</p> <p>Each domain can configure different providers of the &quot;identity&quot;, &quot;authentication&quot;, &quot;access&quot; and &quot;chpass&quot;. For example a configuration in [/etc/sssd/sssd.conf]{.title-ref}</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>[domain/default] </span><span>id_provider = ldap </span><span>auth_provider = ldap </span><span>access_provider = ldap </span><span>chpass_provider = ldap </span></code></pre> <p>The [id_provider]{.title-ref} is the backend of the domain that resolves names and uid/gid numbers to identities.</p> <p>The [auth_provider]{.title-ref} is the backend that validates the password of an identity.</p> <p>The [access_provider]{.title-ref} is the backend that describes if an identity is allowed to access this system or not.</p> <p>The [chpass_provider]{.title-ref} is the backend that password changes and updates are sent to.</p> <p>As you can see there is a lot of flexibility in this design. For example you could use krb5 as the auth provider, but send password changes via ldap.</p> <p>Because of this design SSSD links to and consumes identity management libraries from many other sources such as samba (ad), ldap and kerberos. This means in some limited cases you may need to apply debugging knowledge from the relevant backend to solve an issue in SSSD.</p> <h2 id="common-issues">Common Issues</h2> <h3 id="performance">Performance</h3> <p>In some cases SSSD can be very slow to resolve a user/group on first login, but then becomes &quot;faster&quot; after the login completes. In addition sometimes you may see excessive or high query load on an LDAP server during authentication as well. This is due to an issue with how groups and users are resolved where to resolve a user, you need to resolve it's group memberships. Then each group is resolved, but for unix-tools to display a group you need to resolve it's members. Of course it's members are users and these need resolving ... I hope you can see this is recursive. In some worst cases this can lead to a situation where when a single user logs on, the full LDAP/AD directory is enumerated, which can take minutes in some cases.</p> <p>To prevent this set:</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>ignore_group_members = False </span></code></pre> <p>This prevents groups resolving their members. As a results groups appear to have no members, but users will always display the groups they are member-of. Since almost all applications work using this &quot;member-of&quot; pattern, there are very few negative outcomes from this.</p> <h3 id="cache-clearing">Cache Clearing</h3> <p>SSSD has a local cache of responses from network services. It ships with a cache management tool [sss_cache]{.title-ref}. This allows records to be marked as [invalid]{.title-ref} so that a reload from the network occurs as soon as possible.</p> <p>There are two flaws here. In some cases this appears to have &quot;no effect&quot; where invalid records continue to be served. In addition, the [sss_cache]{.title-ref} tool when called with [-E]{.title-ref} for everything, does not always actually invalidate everything.</p> <p>A common source of advice in these cases is to stop sssd, remove all the content under [/var/lib/sss/db]{.title-ref} (but not the folder itself) and then start sssd.</p> <h3 id="debugging-kerberos">Debugging Kerberos</h3> <p>Kerberos can be notoriously hard to debug. This is because it doesn't have a real verbose/debug mode, at least not obviously. To get debug output you need to set an environment variable.</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>KRB5_TRACE=/dev/stderr kinit user@domain </span></code></pre> <p>This works on <em>any</em> proccess that links to kerberos, so it works on 389-ds, sssd, and many other applications so you can use this to trace what's going wrong.</p> <h2 id="conclusion">Conclusion</h2> <p>That's all for now, I'll probably keep updating this post over time :)</p> Exploring Webauthn Use Cases Mon, 13 Jun 2022 00:00:00 +0000 Unknown https://fy.blackhats.net.au/blog/2022-06-13-exploring-webauthn-use-cases/ https://fy.blackhats.net.au/blog/2022-06-13-exploring-webauthn-use-cases/ <h1 id="exploring-webauthn-use-cases">Exploring Webauthn Use Cases</h1> <p>Webauthn is viewed by many people and companies as the future of authentication on the internet and within our workplaces. It has the support of many device manufacturers, browser vendors and authentication providers.</p> <p>But for Webauthn's lofty goals and promises, as a standard it has many fractured parts. Many of the features it claims at best don't work, at worst, present possible security risks. The standard itself is quite confusing, uses dense and obtuse language, and laid out in a very piecemeal way. This makes it hard to see the full picture to construct a proper security and use cases analysis.</p> <p>As the author of both a relying party ( <a href="https://github.com/kanidm/kanidm">Kanidm</a> ) and the <a href="https://github.com/kanidm/webauthn-rs">Webauthn Library for Rust</a> I want to describe these problems.</p> <p>To understand the issues, we first need to explore how Webauthn works, and then the potential use cases. While not an exhaustive list of all the ways Webauthn could be used, I am trying to cover the ways that I have seen in the wild, and how people have requested we want to use these.</p> <p>Generally, I will try to use <em>accessible</em> language versions of terms, rather than the Webauthn standard terms, as the language in the standard is confusing / misleading - even if you have read the standard multiple times.</p> <h2 id="use-cases">Use Cases</h2> <p>To understand the limitations of Webauthn, we need to examine how Webauthn would be used by an identity provider. The identity provider takes the pieces from Webauthn and their own elements and creates a work flow for the user to interact with. We will turn these into use cases.</p> <p>Remember, the goal of webauthn is to enable all people, from various cultural, social and educational backgrounds to authenticate securely, so it's critical these processes are clear, accessible, and transparent.</p> <p>For the extremely detailed versions of these use cases, see the end of this post.</p> <p>A really important part of these use cases is attestation. Attestation is the same as the little gold star sticker that you found on Nintendo game boxes. It's a &quot;certificate of authenticity&quot;. Without attestation, the authenticator that we are communicating with could be anything. It could be a yubikey, Apple's touchid, a custom-rolled software token, or even a private key you calculated on pen and paper. Attestation is a cryptograhic &quot;certificate of authenticity&quot; which tells us exactly whom produced that device and if it can be trusted.</p> <p>This is really important, because within Webauthn many things are done on the authenticator such as user-verification. Rather than just touching the token, you may have to enter a PIN or use a fingerprint. But the server never sees that PIN or fingerprint - the authenticator just sends us a true/false flag if the verification occured and was valid. So for us to trust this flag (and many others), we need to know that the token is made by someone we trust, so that we know that flag <em>means</em> something.</p> <p>Without this attestation, all we know is that &quot;there is some kind of cryptograhic key that the user can access&quot; and we have no other information about where it might be stored, or how it works. With attestation we can make stronger informed assertions about the properties of the authenticators our users are using.</p> <h3 id="security-token-public">Security Token (Public)</h3> <p>In this use case, we want our authenticator to be a single factor to compliment an existing password. This is the &quot;classic&quot; security key use case, that was originally spawned by U2F. Instead of an authenticator, a TOTP scheme could alternately be used where either the TOTP or authenticator plus the password is sufficient to grant access.</p> <p>Generally in this use case, most identity providers do not care about attestation of the authenticator, what is more important is that some kind of non-password authentication exists and is present.</p> <h3 id="security-token-corporate">Security Token (Corporate)</h3> <p>This is the same as the public use case, except that in many corporations we may want to define a list of trusted providers of tokens. It's important to us here that these tokens have a vetted or audited supply chain, and we have an understanding of &quot;where&quot; the cryptographic material may reside.</p> <p>For this example, we likely want attestation, as well as the ability to ensure these credentials are not recoverable or transferable between authenticators. Resident Key may or may not be required in these cases.</p> <p>Since these are guided by policy, we likely want to have our user interfaces guide our users to register or use the correct keys since we have a stricter list of what is accepted. For example, there is no point in the UI showing a prompt for caBLE (phone authenticator) when we know that only a USB key is accepted!</p> <h3 id="passkey-public">PassKey (Public)</h3> <p>A passkey is the &quot;Apple terminology&quot; for a cryptographic credential that can exist between multiple devices, and potentially even between multiple Apple accounts. This are intended to be a &quot;single factor&quot; replacement to passwords. They can be airdropped and moved between devices, and at the least in their usage with iOS devices, they <em>can</em> perform user verification, but it may not be required for the identity provider to verify this. This is because even as a single factor, these credentials <em>do</em> resolve many of the weaknesses of passwords even if user verification did not occur (and even if it did occur it can not be verified, for reasons we will explore in this post).</p> <p>It is likely we will see Google and Microsoft develop similar. 1Password is already progressing to allow webauthn in their wallets.</p> <p>In this scenario, all we care about is having some kind of credential that is stronger than a password. It's a single factor, and we don't know anything about the make or model of the device. User verification might be performed, but we don't intend to verify if it is.</p> <p>Nothing is really stopping a U2F style token like a yubikey being a passkey, but that relies on the identity provider to allow multiple devices and to have work flows to enrol them across different devices. It's also unclear how this will work from an identity provider when someone has say a Microsoft Surface and an Apple iPhone.</p> <h3 id="passwordless-mfa-public">Passwordless MFA (Public)</h3> <p>In this example, rather than having our authenticator as a single factor, we want it to be truly multifactor. This allows the user to login with nothing but their authenticator, and we have a secure multifactor work flow. This is a stronger level of authentication, where we are verifying not just possession of the private key, but also the identity of who is using it.</p> <p>As a result, we need to strictly verify that the authenticator did a valid user verification.</p> <p>Given that the authenticator is now the &quot;sole&quot; authenticator (even if multi-factor) we are more likely to want attestation here using privacy features granted through indirect attestation. That way we can have a broad list of known good security token providers that we accept. Without attestation we are unable to know if the user verification provided can be trusted.</p> <h3 id="passwordless-mfa-corporate">Passwordless MFA (Corporate)</h3> <p>Again, this is similar to above. We narrow and focus this use case with a stricter attestation list of what is valid. We also again want to strictly control and prevent cryptographic material being moved, so we want to ensure these are not transferrable. We may want resident keys to be used here too since we have a higher level of trust in our devices now too. Again, we also will want to be able to strictly guide UI's due to our knowledge of exactly what devices we accept.</p> <h3 id="usernameless">Usernameless</h3> <p>Usernameless is similar to passwordless but <em>requires</em> resident keys as the username of the account is bound to the key and discovered by the client. Otherwise many of the features of passwordless apply.</p> <p>It's worth noting that due to the complexity and limitations of resident key management <em>it is not feasible</em> for any public service provider to currently use usernameless credentials on a broad scale without significant risk of credential loss. As a result, we limit our use case to corporate only, as they are the only entities in the position to effectively manage these issues.</p> <p>Due to the implementation of passkeys and passwordless in the broader world, the line is blurred between these, so we will assume that passkeys and passwordless may sometimes attempt to be used in a usernameless workflow (for example conditional UI)</p> <h3 id="summary">Summary</h3> <p>Let's assemble a score card now. We'll define the use cases, the features, and what they require and if webauthn can provide them.</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span> Security Token Sec Tok (Corp) PassKey Passwordless PwLess (Corp) </span></code></pre> <hr /> <p><strong>User Verification</strong> no / ??? no / ??? no / ??? required / ??? required / ??? <strong>UV Policy</strong> no / ??? no / ??? no / ??? no / ??? maybe / ??? <strong>Attestation</strong> no / ??? required / ??? no / ??? required / ??? required / ??? <strong>Bound to Device / HW</strong> no / ??? required / ??? no / ??? required / ??? required / ??? <strong>Resident Key</strong> no / ??? maybe / ??? no / ??? maybe / ??? maybe / ??? <strong>UI Selection</strong> maybe / ??? maybe / ??? no / ??? maybe / ??? required / ??? <strong>Update PII</strong> no / ??? no / ??? maybe / ??? maybe / ??? maybe / ??? <strong>Result</strong> ??? ??? ??? ??? ???</p> <p>: Webauthn Score Card</p> <p>Now, I already know some of the answers to these, so lets fill in what we DO know.</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span> Security Token Sec Tok (Corp) PassKey Passwordless PwLess (Corp) </span></code></pre> <hr /> <p><strong>User Verification</strong> no / ??? no / ??? no / ??? required / ??? required / ??? <strong>UV Policy</strong> no / ??? no / ??? no / ??? no / ??? maybe / ??? <strong>Attestation</strong> no / ✅ required / ??? no / ??? required / ??? required / ??? <strong>Bound to Device / HW</strong> no / ✅ required / ??? no / ✅ required / ??? required / ??? <strong>Resident Key</strong> no / ✅ maybe / ??? no / ✅ no / ✅ maybe / ??? <strong>Authenticator Selection</strong> maybe / ??? maybe / ??? no / ??? maybe / ??? required / ??? <strong>Update PII</strong> no / ✅ no / ✅ maybe / ??? maybe / ??? maybe / ??? <strong>Result</strong> ??? ??? ??? ??? ???</p> <p>: Webauthn Score Card</p> <h2 id="the-problems">The Problems</h2> <p>Now lets examine the series of issues that exist within Webauthn, and how they impact our ability to successfully implement the above.</p> <h3 id="authenticator-selection">Authenticator Selection</h3> <p>Today, there is no features in Webauthn that allow an identity provider at registration to pre-indicate what transports are known to be valid for authenticators that are registering. This is contrast to authentication, where a complete list of valid transports can be provided to help the browser select the correct device to use in the authentication.</p> <p>As a result, the only toggle you have is &quot;platform&quot; vs &quot;cross-platform&quot;. Consider we have company issued yubikeys. We know these can only work via USB because that is the model we have chosen.</p> <p>However, during a registration because we can only indicate &quot;cross-platform&quot; it is completely valid for a user to <em>attempt</em> to register say their iPhone via caBLE, or use another key via NFC. The user may then become &quot;confused&quot; why their other keys didn't work for registration - the UI said they were allowed to use it! This is a lack of constraint.</p> <p>This process could be easily streamlined by allowing transports to be specified in registration, but there is resistance to this <a href="https://github.com/w3c/webauthn/issues/1716">from the working group.</a></p> <p>A real world example of this has already occurred, where the email provider <a href="https://www.fastmail.com/">FastMail</a> used specific language around &quot;Security Tokens&quot; including graphics of usb security keys in their documentation. Because of this lack of ability to specify transports in the registration process, once caBLE was released this means that FastMail now has to &quot;rush&quot; to respond to update their UI/Docs to work out how to communicate this to users. They don't have a choice in temporarily excluding this either which may lead to user confusion.</p> <h3 id="user-verification-inconsistent-confusing">User Verification Inconsistent / Confusing</h3> <p>For our security key work flows we would like to construct a situation where the authenticator is a single factor, and the users password or something else is the other factor. This means the authenticator should only require interaction to touch it, and no PIN or biometric is needed.</p> <p>There are some major barriers here sadly. Remember, we want to create a <em>consistent</em> user experience so that people can become confident in the process they are using.</p> <p>The problem is CTAP2.1 - this changes the behaviour of user verification 'discouraged' so that even when you are registering a credential, you always need to enter a PIN or biometrics. However, when authenticating, you never need the PIN or biometric.</p> <p>There is <em>no communication</em> of the fact that the verification is only needed due to it being registration.</p> <p>Surveying users showed about 60% expect when you need to enter your PIN/biometric at registration that it will be <em>required</em> during future authentication. When it is not present during future authentications this confuses people, and trains them that the PIN/biometrics is an inconsistent and untrustworthy dialog. Sometimes it is there - sometimes it is not.</p> <p>When you combine this with the fact that UV=preferred on most RP's is not validating the UV status, we now have effectively trained all our users that user verification can appear and disappear and not to worry about it, it's fine, it's just <em>inconsistent</em> so they never will consider it a threat.</p> <p>It also means that when we try to adopt passwordless it will be <em>harder</em> to convince users this is safe since they may believe that this inconsistent usage of user verification on their authenticators is something that can be easily bypassed.</p> <p>How can you trust that the PIN/biometric means something, when it is sometimes there and sometimes not?</p> <p>This forces us even in our security key work flows to force UV=preferred, and to <em>go beyond the standard</em> to enforce user verification checks are consistent based on their application at registration. This means any CTAP2.1 device, even though it does NOT need a PIN as a single factor authenticator, will require one as a security key to create a consistent user experience and so we can build trust in our user base.</p> <p>At this point since we are effectively forcing UV to always occur, why not just transition to Passwordless?</p> <p>It is worth noting that for <em>almost all identity providers</em> today, that the use of UV=preferred is bypassable, as the user verification is not checked and there is no guidance in the specification to check this. This has affected Microsoft Azure, Nextcloud, and others</p> <p>As a result, the only trustworthy UV policies are required, or preferred with checks that go beyond the standard. As far as I am aware, only Webauthn-RS providers these stricter requirement checks.</p> <p>Discouraged could be used here, but needs user guidance and training to support it due to the inconsistent dialogs with CTAP2.1.</p> <h3 id="user-verification-policy">User Verification Policy</h3> <p>Especially in our passwordless scenarios, as an identity provider we may wish to define policy about what user verification methods we allow from users. For example we may wish for PIN only rather than allowing biometrics. We may also wish to express the policy on the length of the PIN as well.</p> <p>However, nothing in the response an authenticator provides you with this information about what user verification method was used. Instead webauthn defines the <a href="https://www.w3.org/TR/webauthn-3/#sctn-uvm-extension">User Verification Method extension</a> which can allow an identity provider to request the device to provide what UVM was provided.</p> <p>Sadly, nothing supports it in the wild. Experience with Webauthn-RS shows that it is never honoured or provided when requested. This is true of most extensions in Webauthn. For bonus marks did you know all extensions only are answered when you request attestation (this is not mentioned anywhere in the specification!)</p> <p>As a corporate environment, we can kind-of control this through strict attestation lists, but as a public identity provider with attestation it is potentially not possible to know or enforce this due to extensions being widely unsupported and not implemented.</p> <p>The reason this is &quot;kind-of&quot; is that yubikeys support PIN and some models also support biometrics, but there is no distinction in their attestation. This means if we only wanted PIN auth, we could not use yubikeys since there is no way to distinguish these. Additionally, things like minimum PIN length can't be specified since we don't know what manufacturers support this extension. Devices like yubikeys have an inbuilt minimum length of 8, but again we don't know if they'll use PIN given the availability of biometrics.</p> <h3 id="resident-keys-can-t-be-verified">Resident Keys can't be verified</h3> <p>Resident Keys is where we know that the key material lives <em>only</em> within the cryptographic processor of the authenticator. For example, a yubikey by default produces a key wrapped key, where the CredentialID is itself the encrypted private key, and only that yubikey can decrypt that CredentialID to use it as the private key. In very strict security environments, this may present a risk because an attacker <em>could</em> bruteforce the CredentialID to decrypt the private key, allowing the attacker to then use the credential. (It would take millions of years, but you know, some people have to factor that into their risk models).</p> <p>To avoid this, you can request the device create a resident key - a private key that never leaves the device. The CredentialID is just a &quot;reference&quot; to allow the device to look up the Credential but it does not contain the private key itself.</p> <p>The problem is that there is no <em>signal</em> in the attestation or response that indicates if a resident key was created by the device.</p> <p>You can request to find out if this was created with the <a href="https://www.w3.org/TR/webauthn-3/#sctn-authenticator-credential-properties-extension">Credential Properties</a> extension.</p> <p>The devil however, is in the details. Notably:</p> <p><em>&quot;This client registration extension facilitates reporting certain credential properties known by the client&quot;</em></p> <p>A client extension means that this extension is processed by the web browser, and exists in a section of the response that is unsigned, and can not be verified. This means it is open to client side JS tampering and forgery. This means we <em>can not</em> trust the output of this property.</p> <p>As a result, there is <em>no simple way to verify a resident key was created</em>.</p> <p>To make this better, the request to create the resident key <em>is not signed and can be stripped by client side javascript</em>.</p> <p>So any compromised javascript (which Webauthn assumes is trusted) can strip a registration request for a resident key, cause a key-wrapped-key to be created, and then &quot;assert&quot; pretty promise I swear it's resident by faking the response to the extension.</p> <p>The only way to guarantee you have a resident key, is to validate attestation from an authenticator that <em>exclusively</em> makes resident keys (e.g. Apple iOS). Anything else, you can not assert is a true resident key. Even if you subsequently attempt client side discovery of credentials, that is not the same property as the key being resident. This is a trap that many identity providers may not know they are exposed to.</p> <h3 id="resident-keys-can-t-be-administered">Resident Keys can't be administered</h3> <p>To compound the inability to verify creation of a resident key, the behaviour of resident keys (RK) for most major devices is undefined. For example a Yubikey has limited storage for RKs but I have been unable to find documenation about:</p> <ul> <li>How many RKs can exist on an authenticator.</li> <li>If the maximum number is created and we attempt to create more, does it act like a ring buffer and remove the oldest, or simply fail to create more?</li> <li>If it is possible to update usernames or other personal information related to the RKs in this device?</li> <li>Any API's or tooling to list, audit, delete or manage RK's on the device.</li> </ul> <p>These are <em>basic</em> things that are critical for users and administrators, and they simply do not exist. This complete absence of tooling makes RK's effectively useless to most users and deployments since we have no method to manage, audit, modify or delete RK's.</p> <h3 id="bound-to-device-hardware">Bound to Device / Hardware</h3> <p>For the years leading up to 2022, Webauthn and it's design generally assumed a one to one relationship between the hardware of an authenticator, and the public keys it produced. However, that has now changed with the introduction of Apple Passkeys.</p> <p>What is meant by &quot;bound to device&quot; is that given a public key, only a single hardware authenticator exists that has access to the private key to sign something. This generally means that the cryptographic operations, and the private key itself, are only ever known to the secure enclave of the account.</p> <p>Apple's Passkeys change this, allowing a private key to be distributed between multiple devices of an Apple account, but also the ability to transfer the private key to other nearby devices via airdrop. This means the private key is no longer bound to a single physical device.</p> <p>When we design a security policy this kind of detail matters, where some identity providers can accept the benefits of a cryptographic authentication even if the private key is not hardware backed, but other identity providers must require that private keys are securely stored in hardware.</p> <p>The major issue in Webauthn is that the specification does not really have the necessary parts in place to manage these effectively.</p> <p>As an identity provider there is no way to currently indicate that you require a hardware bound credential (or perhaps you want to require passkeys only!). Because of this lack of control, Apple's implementation relies on another signal - a request for attestation.</p> <p>If you do <em>not</em> request attestation, a passkey is created.</p> <p>If you do request attestation (direct or indirect), a hardware bound key is created.</p> <p>When the credential is created, there are a new set of &quot;backup state&quot; bits that can indicate if the credential can be moved between devices. These are stored in the same set of bits that stores user verification bits, meaning that to trust them, you need attestation (which Apple can't provide!). At the very least, the attested Apple credentials that are hardware bound, do correctly show they are <em>not</em> backup capable and are still resident keys.</p> <p>Because of this, I expect to see that passkeys and related technology is treated in the manner as initially described - a single-factor replacement to passwords. Where you need stronger MFA in the style of a passwordless credential, it will not currently be possible to achieve this with Apple Passkeys.</p> <p>It's worth noting that it's unclear how other vendors will act here. Some may produce passkeys that are attested, meaning that reliance on the backup state bits will become more important, but there is also a risk that vendors will not implement this correctly.</p> <p>Importantly some testing in pre-release versions showed that if passkeys are enabled, and you request an attested credential, the registration fails blocking the bound credential creation. This will need retesting to be sure of the behaviour in the final iOS 16 release, but this could be a show stopper for BYOD users if not fixed. (20220614: We have confirmed that passkeys do block the creation of attested device bound credentials).</p> <h2 id="conclusion">Conclusion</h2> <ul> <li> <p>⚠️ - risks exist</p> </li> <li> <p>✅ - works</p> </li> <li> <p>❌ - broken/untrustworthy</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span> Security Token Sec Tok (Corp) PassKey Passwordless PwLess (Corp) </span></code></pre> </li> </ul> <hr /> <p><strong>User Verification</strong> no / ⚠️ no / ⚠️ no / ⚠️ required / ✅ required / ✅ <strong>UV Policy</strong> no / ✅ no / ✅ no / ✅ no / ✅ maybe / ❌ <strong>Attestation</strong> no / ✅ required / ⚠️ no / ✅ required / ⚠️ required / ⚠️ <strong>Bound to Device / HW</strong> no / ✅ required / ⚠️ no / ✅ required / ⚠️ required / ⚠️ <strong>Resident Key</strong> no / ✅ maybe / ❌ no / ✅ no / ✅ maybe / ❌ <strong>Authenticator Selection</strong> maybe / ❌ maybe / ❌ no / ✅ maybe / ❌ required / ❌ <strong>Update PII</strong> no / ✅ no / ✅ maybe / ❌ maybe / ❌ maybe / ❌ <strong>Result</strong> ⚠️ 1, 2, 7 ⚠️ 1, 2, 4, 5, 6, 7 ⚠️ 1, 2, 8 ⚠️ 4, 5, 7, 8 ⚠️ 4, 5, 6, 7, 8</p> <p>: Webauthn Score Card</p> <ol> <li>User Verification in discouraged may incorrectly request UV, training users that UV prompts are &quot;optional&quot;.</li> <li>UV preferred, is bypassable in almost all implementations.</li> <li>No method to request a UV policy including min PIN length or UV classes.</li> <li>Existence of PassKeys on the device account, WILL prevent attested credentials from being created.</li> <li>Currently relies on vendor specific attestation behaviour.</li> <li>No way to validate a resident key is created without assumed vendor specific behaviours, or other out of band checks.</li> <li>Unable to request constraints for authenticators that are used in the interaction.</li> <li>Vendors often do not provide the ability to update PII on resident keys if used in these contexts</li> </ol> <p>A very interesting take away from this however, is that &quot;Passkeys&quot; that Apple have created, are actually identical to &quot;Security Tokens&quot; in how they operate and are validated, meaning that for all intents and purposes they are the same scenario just with or without a password as the MFA element.</p> <p>As we can see, from our use cases all of the scenarios have some kind of issues. They vary in severity and whom the issue affects, but they generally are all subtle and may have implications on identity providers. Generally the &quot;trend&quot; from these issues though, is that it feels like the Webauthn WG have abandoned authenticators as &quot;security tokens&quot; and are pushing more toward Passkeys as Single Factor or Passwordless scenarios. This is probably &quot;a good thing&quot;, but it's not been communicated clearly and there are still issues that exist in the Passkey and Passwordless scenarios.</p> <h2 id="bonus-other-skeletons">Bonus - Other Skeletons</h2> <h3 id="javascript-is-considered-trusted">Javascript is considered trusted</h3> <p>Because Javascript is considered trusted, a large number of properties of Webauthn in its communication are open to tampering which means that they infact, can not be trusted. Because we can't trust the JS or the user not to tamper with their environment, we need to only trust properties that are from the browser or authenticator, and then signed. As a result, regardless of whom we are, we need to assume this in our threat models that anything on a webpage, can and will be altered. If the browser or authenticator are compromised, we have different issues, and different defences.</p> <h3 id="insecure-crypto">Insecure Crypto</h3> <p>Windows Hello especially relies on TPM's that have their attestation signed with sha1. Sha1 is considered broken, meaning that it could be possible to forge attestations trivially of these credentials. Newer TPM's may not have this limitation.</p> <h3 id="unclear-what-is-is-not-security-property">Unclear what is / is not security property</h3> <p>A large limitation of Webauthn is that it is unclear what <em>is</em> or <em>is not</em> a security property within the registration and authentication messages. For now, we'll focus on the registration. This is presented with all the options and structures expanded that are relevant. Imagine you are an identity provider implementing a webauthn library and you see the following.</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>PublicKeyCredentialCreationOptions { </span><span> rp = &quot;relying party identifier&quot; </span><span> user { </span><span> id = &quot;user id&quot; </span><span> displayName = &quot;user display name&quot; </span><span> } </span><span> challenge = [0xAB, 0xCD, ... ] </span><span> PublicKeyCredentialParameters = [ </span><span> { </span><span> type = &quot;public-key&quot;; </span><span> alg =&quot;ECDSA w/ SHA-256&quot; | ... | &quot;RSASSA-PKCS1-v1_5 using SHA-1&quot; </span><span> }, ... </span><span> ] </span><span> timeout = 60000 </span><span> excludeCredentials = [ </span><span> { </span><span> type = &quot;public-key&quot; </span><span> id = [0x00, 0x01, ... ] </span><span> transports = [ &quot;usb&quot; | &quot;ble&quot; | &quot;internal&quot; | &quot;nfc&quot;, ... ] </span><span> } </span><span> ] </span><span> authenticatorSelection = { </span><span> authenticatorAttachment = &quot;platform&quot; | &quot;cross-platform&quot; </span><span> userVerification = &quot;discouraged&quot; | default=&quot;preferred&quot; | &quot;required&quot; </span><span> requireResidentKey = boolean </span><span> }; </span><span> attestation = default=&quot;none&quot; | &quot;indirect&quot; | &quot;direct&quot; | &quot;enterprise&quot; </span><span> extensions = ... </span><span>}; </span></code></pre> <p>Now, reading this structure, which elements do you think are security properties that you can rely upon to be strictly enforced, and have cryptographic acknowledgement of that being enforced?</p> <p>Well, only the following are signed cryptographically by the authenticator:</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>PublicKeyCredentialCreationOptions { </span><span> rp = &quot;relying party identifier&quot; </span><span> challenge = [0xAB, 0xCD, ... ] </span><span>} </span></code></pre> <p>We can assert the credential algorithm used by checking it (provided we are webauthn level 2 compliant or greater). And we can only check if the userVerification happened or not through the returned attestation. This means the following aren't signed (for the aware, extensions are something we'll cover seperately).</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>PublicKeyCredentialCreationOptions { </span><span> user { </span><span> id = &quot;user id&quot; </span><span> displayName = &quot;user display name&quot; </span><span> } </span><span> timeout = 60000 </span><span> excludeCredentials = [ </span><span> { </span><span> type = &quot;public-key&quot; </span><span> id = [0x00, 0x01, ... ] </span><span> transports = [ &quot;usb&quot; | &quot;ble&quot; | &quot;internal&quot; | &quot;nfc&quot;, ... ] </span><span> } </span><span> ] </span><span> authenticatorSelection = { </span><span> authenticatorAttachment = &quot;platform&quot; | &quot;cross-platform&quot; </span><span> requireResidentKey = boolean </span><span> }; </span><span>}; </span></code></pre> <p>This means that from our registration we can not know or assert:</p> <ul> <li>If an excluded credential was used or not</li> <li>If a resident key was really created</li> <li>If the created credential is platform or cross platform</li> </ul> <h3 id="extensions">Extensions</h3> <p>Most extensions are not implemented at all in the wild, making them flat out useless.</p> <p>Many others are client extensions, meaning they are run in your browser and are not signed, and can be freely tampered with without verification as javascript is trusted.</p> <h2 id="extremely-detailed-use-cases">Extremely Detailed Use Cases</h2> <p>The use cases we detail here are significantly richer and more detailed than the ones in the <a href="https://www.w3.org/TR/webauthn-3/#sctn-use-cases">specification</a> (2022-04-13).</p> <p>Each workflow has two parts. A registration (on-boarding) and authentication. Most of the parameters for webauthn revolve around the behaviour at registration, with authentication being a much more similar work flow regardless of credential type.</p> <h3 id="security-token-public-1">Security Token (Public)</h3> <p>Registration:</p> <ol> <li>The user indicates they wish to enroll a security token</li> <li>The identity provider issues a challenge</li> <li>The browser lists which authenticators attached to the device <em>could</em> be registered</li> <li>The user interacts with the authenticator (<em>note</em> a pin should not be requested, but fingerprint is okay since it's &quot;transparent&quot;)</li> <li>The authenticator releases the signed public key</li> <li>The authenticator is added to the users account</li> </ol> <p>Authentication:</p> <ol> <li>The user enters their username</li> <li>The user provides their password and it is validated (<em>note</em> we could do this after webauthn)</li> <li>The user indicates they wish to use a security token</li> <li>The identity provider issues a webauthn challenge, limited by the list of authenticators and transports we know are valid for the authenticators associated.</li> <li>The browser offers the list of authenticators that can proceed</li> <li>The user interacts with the authenticator (<em>note</em> a pin should not be requested, but fingerprint is okay since it's &quot;transparent&quot;)</li> <li>The authenticator releases the signature</li> </ol> <h3 id="security-token-corporate-1">Security Token (Corporate)</h3> <p>Registration:</p> <ol> <li>The user indicates they wish to enroll a security token</li> <li>The identity provider issues a challenge, with a list of what transports of <em>known</em> approved authenticators exist that could be used.</li> <li>The browser lists which authenticators attached to the device <em>could</em> be registered, per the transport list</li> <li>The user interacts with the authenticator (<em>note</em> a pin should not be requested, but fingerprint is okay since it's &quot;transparent&quot;)</li> <li>The authenticator releases the signed public key</li> <li>The identity provider examines the attestation and asserts it is from a trusted manufacturer</li> <li>The identity provider examines the enrollment, and asserts it is bound to the hardware (IE not a passkey/backup)</li> <li>The authenticator is added to the users account</li> </ol> <p>Authentication:</p> <ol> <li>As per Security Token (public)</li> </ol> <h3 id="passkey-public-1">PassKey (Public)</h3> <p>Registration:</p> <ol> <li>The user indicates they wish to enroll a token</li> <li>The identity provider issues a challenge</li> <li>The browser lists which authenticators attached to the device <em>could</em> be registered</li> <li>The user interacts with the authenticator (<em>note</em> a pin should not be requested, but fingerprint is okay since it's &quot;transparent&quot;)</li> <li>The authenticator releases the signed public key</li> <li>The authenticator is added to the users account</li> </ol> <p>Authentication:</p> <ol> <li>The user enters their username</li> <li>The identity provider issues a webauthn challenge, limited by the list of authenticators and transports we know are valid for the authenticators associated.</li> <li>The browser offers the list of authenticators that can proceed</li> <li>The user interacts with the authenticator (<em>note</em> a pin should not be requested, but fingerprint is okay since it's &quot;transparent&quot;)</li> <li>The authenticator releases the signature</li> </ol> <h3 id="passwordless-public">Passwordless (Public)</h3> <p>Registration:</p> <ol> <li>The user indicates they wish to enroll a security token</li> <li>The identity provider issues a challenge</li> <li>The browser lists which authenticators attached to the device <em>could</em> be registered</li> <li>The user interacts with the authenticator - user verification MUST be provided i.e. pin or biometric.</li> <li>The authenticator releases the signed public key</li> <li>The identity provider asserts that user verification occured</li> <li>(Optional) The identity provider examines the attestation and asserts it is from a trusted manufacturer</li> <li>The authenticator is added to the users account</li> </ol> <p>Authentication:</p> <ol> <li>The user enters their username</li> <li>The identity provider issues a webauthn challenge</li> <li>The browser offers the list of authenticators that can proceed</li> <li>The user interacts with the authenticator - user verification MUST be provided i.e. pin or biometric.</li> <li>The authenticator releases the signature</li> <li>The identity provider asserts that user verification occured</li> </ol> <h3 id="passwordless-corporate">Passwordless (Corporate)</h3> <p>Registration:</p> <ol> <li>The user indicates they wish to enroll a security token</li> <li>The identity provider issues a challenge, with a list of what transports of <em>known</em> approved authenticators exist that could be used.</li> <li>The browser lists which authenticators attached to the device <em>could</em> be registered, per the transport list</li> <li>The user interacts with the authenticator - user verification MUST be provided i.e. pin or biometric.</li> <li>The authenticator releases the signed public key</li> <li>The identity provider examines the attestation and asserts it is from a trusted manufacturer</li> <li>(Optional) The identity provider asserts that a resident key was created</li> <li>The identity provider examines the enrollment, and asserts it is bound to the hardware (IE not a passkey/backup)</li> <li>The identity provider asserts that user verification occured</li> <li>(Optional) The identity provider asserts the verification method complies to policy</li> <li>The authenticator is added to the users account</li> </ol> <p>Authentication:</p> <ol> <li>As per Passwordless (public)</li> </ol> <h3 id="usernameless-1">Usernameless</h3> <p>Registration</p> <ol> <li>The user indicates they wish to enroll a security token</li> <li>The identity provider issues a challenge, with a list of what transports of <em>known</em> approved authenticators exist that could be used.</li> <li>The browser lists which authenticators attached to the device <em>could</em> be registered, per the transport list</li> <li>The user interacts with the authenticator - user verification MUST be provided i.e. pin or biometric.</li> <li>The authenticator releases the signed public key</li> <li>The identity provider examines the attestation and asserts it is from a trusted manufacturer</li> <li>The identity provider asserts that a resident key was created</li> <li>The identity provider examines the enrollment, and asserts it is bound to the hardware (IE not a passkey/backup)</li> <li>The identity provider asserts that user verification occured</li> <li>(Optional) The identity provider asserts the verification method complies to policy</li> <li>The authenticator is added to the users account</li> </ol> <p>Authentication:</p> <ol> <li>The identity provider issues a webauthn challenge</li> <li>The browser offers the list of authenticators that can proceed</li> <li>The user interacts with the authenticator - user verification MUST be provided i.e. pin or biometric.</li> <li>The authenticator releases the signature</li> <li>The identity provider asserts that user verification occured</li> <li>The identity provider extracts and uses the provided username that was supplied</li> </ol> Enable caBLE on your iPhone for testing Mon, 04 Apr 2022 00:00:00 +0000 Unknown https://fy.blackhats.net.au/blog/2022-04-04-enable-cable-on-your-iphone-for-testing/ https://fy.blackhats.net.au/blog/2022-04-04-enable-cable-on-your-iphone-for-testing/ <h1 id="enable-cable-on-your-iphone-for-testing">Enable caBLE on your iPhone for testing</h1> <p>caBLE allows a nearby device (such as your iPhone) to be used an a webauthn authenticator. Given my work on <a href="https://github.com/kanidm/webauthn-rs">WebauthnRS</a> I naturally wanted to test this! When I initially tried to test caBLE with webauthn via my iPhone, I recieved an error that the operation wasn't available at this time. There was no other information available.</p> <h2 id="debugging">Debugging</h2> <p>After some digging into Console.app, I found the log message from AuthenticationServicesAgent which stated:</p> <p><em>&quot;Syncing platform authenticator must be enabled to register a platform public key credential; this can be enabled in Settings &gt; Developer.&quot;</em></p> <h2 id="enabling-developer-settings">Enabling Developer Settings</h2> <p>Run Xcode.app on your mac. On your iPhone close and reopen settings. Then search for &quot;developer&quot;.</p> <p>Inside of that menu enable [Syncing Platform Authenticator]{.title-ref} and [Additional Logging]{.title-ref} under [PassKit]{.title-ref}.</p> <p>After that you should be able to test caBLE!</p> Documentation PR's Welcome - Why Docs Are Not A Beginner Friendly Task Tue, 15 Mar 2022 00:00:00 +0000 Unknown https://fy.blackhats.net.au/blog/2022-03-15-documentation-pr-s-welcome-why-docs-are-not-a-beginner-friendly-task/ https://fy.blackhats.net.au/blog/2022-03-15-documentation-pr-s-welcome-why-docs-are-not-a-beginner-friendly-task/ <h1 id="documentation-pr-s-welcome-why-docs-are-not-a-beginner-friendly-task">Documentation PR's Welcome - Why Docs Are Not A Beginner Friendly Task</h1> <p>Recently I was reporting a usability issue with a library, mainly related to it's confusing or absent documentation. A friend of mine saw the exchange and commented (quite accurately) that it went along the lines of:</p> <ul> <li>Me: This library should improve it's documentation</li> <li>Project: PR's welcome</li> <li>Me: I can't write the docs because I don't know how this works without documentation</li> </ul> <p>My friend also commented <em>&quot;[this] is probably the realest interaction I've seen in a while.&quot;</em> and <em>&quot;It kinda shakes up the idea that if people want something in OSS they should do it themselves, and that docs are the easiest way to contribute&quot;</em>.</p> <p>I completely agree with these observations.</p> <p>Documentation writing is not a task for a drive by contributor, or a beginner. Documentation writing is a skill in and of itself, that requires many things. From my perspective, I believe documentation writing requires:</p> <ul> <li>Linguistic ability - to express ideas clearly to a diverse audience of readers.</li> <li>Emotional intelligence - to empathise with their audience and to think &quot;what would help them in these words&quot;?</li> <li>Technical knowledge - the understanding of the problem space to know how to write the correct documentation.</li> </ul> <p>Generally, when someone is a beginner to a project they lack an understanding of the project audience so they aren't able to empathise with &quot;what could be useful for a reader&quot;. They also tend to lack the depth and breath of technical knowledge to know <em>what</em> to write for the documentation since by definition, they are a newcommer to this project.</p> <p>A great way to connect with a beginner is to listen to their frustrations and what challenges the encountered with your project. First, connect how that frustration could be either a failing of the user interface and it's design (even an API is a user interface). Second, only once you have eliminated design limitations, then consider it to be a documentation problem. As the project contributor, you are in a better position to write documentation than a beginner.</p> <p>Telling someone, especially a beginner, &quot;docs PR's welcome&quot; is both belitting to the skill that is technical writing, but also generally considered in opensource to mean &quot;I don't give a shit about your problem, fuck off&quot;.</p> How CTAP2.0 made UserVerification even more confusing Wed, 19 Jan 2022 00:00:00 +0000 Unknown https://fy.blackhats.net.au/blog/2022-01-19-how-ctap2-0-made-userverification-even-more-confusing/ https://fy.blackhats.net.au/blog/2022-01-19-how-ctap2-0-made-userverification-even-more-confusing/ <h1 id="how-ctap2-0-made-userverification-even-more-confusing">How CTAP2.0 made UserVerification even more confusing</h1> <p>I have previously written about how <a href="../../../2020/11/21/webauthn_userverificationpolicy_curiosities.html">Webauthn introduces a false sense of security</a> with how it manages UserVerification (UV) by default. To summarise, when you request &quot;preferred&quot; which means &quot;perform UV if possible&quot;, it can be bypassed since relying parties's (RP) do <em>not</em> check if UV was actually performed, and Webauthn makes no recommendations on how to store credentials in a manner that allows future checking to ensure UV is requested or validated correctly.</p> <p>From this, in Webauthn-RS we made the recommendation that you use either &quot;required&quot; to enforce all credentials have performed UV, or &quot;discouraged&quot; to request that <em>no</em> UV is performed by credentials during authentication or registration.</p> <p>At the same time, in the Webauthn-RS project we begun to store two important pieces of credential metadata beyond the Webauthn specification - the result of UV from registration, and the policy that was requested at the time of registration. We did this because we had noticed there were classes of credentials, that even in &quot;discouraged&quot; would always verify themself at registration and authentication. Because of this property, we would enforce that since UV was performed at registration, we could continue to enforce UV on a per credential basis to detect possible credential compromise, and to further strengthen the security of credentials used with Webauthn-RS.</p> <p>This created 3 workflows:</p> <ul> <li>Required - At registration and authentication UV is always required</li> <li>Discouraged + no UV - At registration and authentication UV is never required</li> <li>Discouraged + always UV - At registration and authentication UV is always required</li> </ul> <h2 id="a-bug-report">A bug report ...</h2> <p>We recieved a bug that an authenticator was failing to work with Webauthn-RS, because at registration it would always force UV, but during authentication it would <em>never</em> request UV. This was triggering our inconsistent credential detection, indicating the credential was possibly compromised.</p> <p>In this situation, the authenticator used an open-source firmware, so I was able to look at the source and identify the programming issue. During registration UV is <em>always</em> required, but during &quot;discouraged&quot; in authentication it's <em>never</em> required matching the reported bug.</p> <p>The author of the library then directed me to the fact that in <a href="https://fidoalliance.org/specs/fido-v2.0-ps-20190130/fido-client-to-authenticator-protocol-v2.0-ps-20190130.html#authenticatorMakeCredential">CTAP2.0</a> this behaviour is <em>enshrined in the specification</em>.</p> <h2 id="why-is-this-behaviour-bad">Why is this behaviour bad?</h2> <p>I performed a quick poll on twitter, and asked about 5 non-technical friends about this. The question I asked was:</p> <p><em>You go to a website, and you're asked to setup a yubikey. When you register the key you're asked for a pin. Do you now expect the pin to be required when you authenticate to that website with that yubikey now?</em></p> <p>From the 31 votes on twitter, the result was 60% (21 / 30) that &quot;yes&quot; this PIN will always be required. From the people I asked directly, they all responded &quot;yes&quot;. (This is in no way an official survey or significant numbers, but it's an initial indication)</p> <p>Humans expect things to behave in a <em>consistent</em> manner. When you take an action one time, something will always continue to behave in that way. The issue we are presented with in this situation is that CTAP2.0 fundamentally breaks this association by changing the behaviour between registration and authentication. It also is not communicated that the different is registration vs authentication, or even <em>why</em> this behaviour is changed.</p> <p>As a result, this confuses users (&quot;Why is my pin not always required?!&quot;) and this can at worst cause users to be apathetic about the UV check, where it could be downgraded from &quot;required/preferred&quot; to &quot;discouraged&quot; and the user would not notice or care about &quot;why is this different?&quot;. Because RP's that strictly follow the Webauthn specification are open to UV bypass, CTAP2.0 in this case has helped to open the door for users to be tricked into this.</p> <p>The other issue is that for a library like Webauthn-RS we lose the ability to detect credential compromise or attempts to bypass UV when in discouraged, since now UV is not consistently enforced across all classes of authenticators.</p> <h2 id="can-it-be-fixed">Can it be fixed?</h2> <p>No. There are changes in CTAP2.1 that can set the token to be &quot;always verified&quot; and for extensions to be sent that always enforce UV of that credential, but none of these assist the CTAP2.0 case where none of these elements exist.</p> <p>As an RP library author we have to assume and work out ways to interact with credentials that are CTAP2.0_pre, CTAP2.0, CTAP2.1, vendor developed and more. We have to find a way to use the elements at hand to create a consistent user interface, that also embed security elements that can not be bypassed or downgraded.</p> <p>I spent a lot of time thinking about how to resolve this, but I can only conclude that CTAP2.0 has made &quot;discouraged&quot; less meaningful by adding in this confusing behaviour.</p> <h2 id="is-this-the-end-of-the-world">Is this the end of the world?</h2> <p>Not at all. But it does undermine users trust in the systems we are building, where people may end up believing that UV is pointless and never checked. There are a lot of smart bad-people out there and they may utilise this in attacks (especially when combined with the fact that RP's who strictly follow the Webauthn standard are already open to UV bypass in many cases).</p> <p>If the goal we have is to move to a passwordless world, we need people to trust their devices behave in a manner that is predictable and that they understand. By making UV sometimes there, sometimes not, it will be a much higher barrier to convince people they can trust these devices as a self contained multifactor authenticator.</p> <h2 id="i-use-an-authentication-provider-what-can-i-do">I use an authentication provider, what can I do?</h2> <p>If possible, setup your authentication provider to have UV required. This will cause some credentials to no longer work in your environment, but it will ensure that every authenticator has a consistent experience. In most cases, your authentication provider is likely to be standards compliant, and will not perform the extended verification discussed below meaning that &quot;preferred&quot; is bypassable, and &quot;discouraged&quot; can have inconsistent UV requests from users.</p> <h2 id="what-can-an-rp-do">What can an RP do?</h2> <p>Because of this change, there are really only three workflows now that are actually consistent for users where we can enforce UV properties are observed correctly.</p> <ul> <li>Required - At registration and authentication UV is always required</li> <li>Preferred + with UV - At registration and authentication UV is always required</li> <li>Preferred + no UV - At registration and authentication UV should not be required</li> </ul> <p>The way that this is achieved is an extension to the Webauthn specification. When you register a credential you <em>must</em> store the state of the UV boolean at registration, and you <em>must</em> store the policy that was requested at registration. During authentication the following is checked in place of the webauthn defined UV check:</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>if credential.registration_policy == required OR authentication.policy == required { </span><span> assert(authentication.uv == true) </span><span>} else if credential.registration_policy == preferred AND credential.registration_uv == true { </span><span> // We either sent authentication.policy preferred or discouraged, but the user registered </span><span> // with UV so we enforce that behaviour. </span><span> assert(authentication.uv == true) </span><span>} else { </span><span> // Do not check uv. </span><span>} </span></code></pre> <p>There is a single edge case in this work flow - since we now send &quot;preferred&quot; it's possible that a credential that registered without UV (IE via Firefox which doesn't support CTAP2.0_pre or greater) will be moved to using a platform that does support CTAP2.0_pre or greater, and it will begin to request UV. It is however possible in this scenario that once the credential begins to provide UV we can then store the credential.uv as true and enforce that for future authentications.</p> <p>The primary issue with this is that we will begin to ask for the user's PIN more often with credentials which may lead to frustration. Biometrics this is less of a concern as the &quot;touch&quot; action is always required anyway. However I think this is acceptable since it's more important for a consistent set of behaviours to exist.</p> <p>Previously I have stated that &quot;preferred&quot; should not be used since it is bypassable, but with the extensions to Webauthn above where policy and uv at registration are stored and validated, preferred gains a proper meaning and can be checked and enforced.</p> <h2 id="conclusion">Conclusion</h2> <p>In the scenarioes where &quot;discouraged&quot; and &quot;preferred&quot; may be used, UV is meaningless in the current definition of the Webauthn specification when paired with the various versions of CTAP. It's merely a confusing annoyance that we present to users seemingly at random, that is trivially bypassed, adds little to no security value and at worst undermines user trust in the systems we are trying to build.</p> <p>When we are building authentication systems, we must always think about and consider the humans who will be using these systems, and the security properties that we actually provide in these systems.</p> Nextcloud - Unable to Open Photos Library Tue, 21 Dec 2021 00:00:00 +0000 Unknown https://fy.blackhats.net.au/blog/2021-12-21-nextcloud-unable-to-open-photos-library/ https://fy.blackhats.net.au/blog/2021-12-21-nextcloud-unable-to-open-photos-library/ <h1 id="nextcloud-unable-to-open-photos-library">Nextcloud - Unable to Open Photos Library</h1> <p>I noticed since macos 11.6.2 that Nextcloud has been unable to sync my photos library. Looking into this error in Console.app I saw:</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>error kernel System Policy: Nextcloud(798) deny(1) file-read-data /Users/william/Pictures/Photos Library.photoslibrary </span></code></pre> <p>It seems that Nextcloud is not <em>sandboxed</em> which means that macos enforces stricter permissions on what this can or can not access, which is what prevented the photos library from syncing.</p> <p>To resolve this you can go to System Preferences -&gt; Security and Privacy -&gt; Privacy -&gt; Full Disk Access and then grant Nextcloud.app full disk access which will allow it to read the filesystem.</p> <p>I tried to allow this via the files and folders access but I was unable to add/remove new items to this list.</p> Transactional Operations in Rust Sun, 14 Nov 2021 00:00:00 +0000 Unknown https://fy.blackhats.net.au/blog/2021-11-14-transactional-operations-in-rust/ https://fy.blackhats.net.au/blog/2021-11-14-transactional-operations-in-rust/ <h1 id="transactional-operations-in-rust">Transactional Operations in Rust</h1> <p>Earlier I was chatting to Yoshua, the author of this <a href="https://blog.yoshuawuyts.com/async-cancellation-1/">async cancellation</a> blog about the section on halt-safety. The blog is a great read so I highly recommend it! The section on halt-safety is bang on correct too, but I wanted to expand on this topic further from what they have written.</p> <h2 id="memory-safety-vs-application-safety">Memory Safety vs Application Safety</h2> <p>Yoshua provides the following code example in their blog:</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>// Regardless of where in the function we stop execution, destructors will be </span><span>// run and resources will be cleaned up. </span><span>async fn do_something(path: PathBuf) -&gt; io::Result&lt;Output&gt; { </span><span> // 1. the future is not guaranteed to progress after instantiation </span><span> let file = fs::open(&amp;path).await?; // 2. `.await` and 3. `?` can cause the function to halt </span><span> let res = parse(file).await; // 4. `.await` can the function to halt </span><span> res // 5. execution has finished, return a value </span><span>} </span></code></pre> <p>In the example, we can see that at each await point the async behaviour could cause the function to return. This would be similar to the non-async code of:</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>fn do_something(path: PathBuf) -&gt; io::Result&lt;Output&gt; { </span><span> let file = fs::open(&amp;path)?; // 1. `?` will return an Err if present </span><span> let res = parse(file); // </span><span> res // 2. res may be an Err at this point. </span><span>} </span></code></pre> <p>In this example we can see that both cancelation <em>or</em> and Err condition could both cause our function to return, regardless of async or not. In this example, since there are no side-effects, it's not a big deal, but let's consider a different example that does have side-effects:</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>fn do_something(path: PathBuf, files_read_counter: &amp;Mutex&lt;u64&gt;) -&gt; io::Result&lt;Output&gt; { </span><span> let mut guard = files_read_counter.lock(); </span><span> let file = fs::open(&amp;path)?; // 1. `?` will return an Err if present </span><span> guard += 1; // </span><span> let res = parse(file); // </span><span> res // 2. res may be an Err at this point. </span><span>} </span></code></pre> <p>This is a nonsensical example, but it illustrates the point. The files read is incremented <em>before</em> we know that the success occured. Even though this is memory safe, it's created an inconsistent data point that is not reflective of the true state. It's trivial to resolve when we look at this (relocation of the guard increment), but in a larger example it may not be as easy:</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>// This is more psuedo rust vs actual rust for simplicities sake. </span><span>fn do_something(...) -&gt; Result&lt;..., ...&gt; { </span><span> let mut guard = map.lock(); </span><span> guard </span><span> .values_mut() </span><span> .try_for_each(|(k, v)| { </span><span> v.update(...) </span><span> }) </span><span>} </span></code></pre> <p>In our example we have a fallible value update function, which is inside our locked datastructure. It would be very simple to see a situation where while updating some values, an error is encountered somewhere into the set, and then an Err returned. But what happens to the entries we <em>did</em> update? Since we return from the Err here, the guard will be dropped, and the lock successfully released, meaning that we have only partially updated our map in this situation. This kind of behaviour can still be defended against as a programmer, but it requires us as humans to bear this cognitive load to ensure our application is behaving safely. This is the difference between memory and application safety.</p> <h2 id="databases">Databases</h2> <p>Databases have confronted this problem for many decades now, and a key logical approach is ACID compliance:</p> <ul> <li>Atomicity - each operation is a single unit that fails or succeeds together</li> <li>Consistency - between each unit, the data always moves from a valid state to another valid state</li> <li>Isolation - multiple concurrent operations should behave as though they are executed in serial</li> <li>Durability - the success of a unit is persisted in the event of future errors IE power-loss</li> </ul> <p>For software, we tend to care more for ACI in this example, but of course if we are writing a database in Rust, it would be important to consider D.</p> <p>When we look at our examples from before, these both fail the atomicity and consistency checks (but they are correctly isolated due to the mutex which enforces serialisation).</p> <h2 id="acid-in-software">ACID in Software</h2> <p>If we treat a top level functional call as our outer operation, and the inner functions as the units comprising this operation, then we can start to look at calls to functions as a transactional entity, where the call to a single operation either succeeds or fails, and the functions within that are [unsafe]{.title-ref} (aka [spicy]{.title-ref} 🌶 ) due to the fact they can create inconsistent states. We want to write our functions in a way that [spicy]{.title-ref} functions can only be contained within operations and creates an environment where either the full operation succeeds or fails, and then ensures that consistency is maintained.</p> <p>An approach that can be used is software transactional memory. There are multiple ways to structure this, but copy-on-write is a common technique to achieve this. An example of a copy-on-write cell type is in <a href="https://crates.io/crates/concread">concread</a>. This type allows for ACI (but not D) compliance.</p> <p>Due to the design of this type, we can seperate functions that are acquiring the guard (operations) and the functions that comprise that operation as they are a passed a transaction that is in progress. For example:</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>// This is more psuedo rust vs actual rust for simplicities sake. </span><span>fn update_map(write_txn: &amp;mut WriteTxn&lt;Map&lt;..., ...&gt;&gt;) -&gt; Result&lt;..., ...&gt; { </span><span> write_txn </span><span> .values_mut() </span><span> .try_for_each(|(k, v)| { </span><span> v.update(...) </span><span> }) </span><span>} </span><span> </span><span>fn do_something(...) -&gt; Result&lt;..., ...&gt; { </span><span> let write_txn = data.write(); </span><span> let res = update_map(write_txn)?; </span><span> write_txn.commit(); </span><span> Ok(res) </span><span>} </span></code></pre> <p>Here we can already see a difference in our approach. We know that for update_map to be called we must be within a transaction - we can not &quot;hold it wrong&quot;, and the compiler checks this for us. We can also see that we invert drop on the write_txn guard from &quot;implicit commit&quot; to a drop being a rollback operation. The commit only occurs <em>explicitly</em> and takes ownership of the write_txn preventing it being used any further without a new transaction. As a result in our example, if update_map were to fail, we would implicitly rollback our data.</p> <p>Another benefit in this example is async, thread and concurrency safety. While the write_txn is held, no other writes can proceed (serialised). Readers are also isolated and guaranteed that their data will not chainge for the duration of that operation (until a new read is acquired). Even in our async examples, we would be able to correctly rollback during an async cancelation or error condition.</p> <h2 id="future-work">Future Work</h2> <p>At the moment the copy on write structures in concread only can protect single datastructures, so for more complex data type you end up with a struct containing many transactional cow types. There is some work going on to allow the creation of a manager that can allow arbitary structures of multiple datatypes to be protected under a single transaction manager, however this work is extremely [unsafe]{.title-ref} though due to the potential for memory safety violations with incorrect construction of the structures. For more details see the <a href="https://docs.rs/concread/0.2.19/concread/internals/index.html">concread internals</a> , <a href="https://docs.rs/concread/0.2.19/concread/internals/lincowcell/trait.LinCowCellCapable.html">concread linear cowcell</a> and, <a href="https://github.com/kanidm/concread/blob/master/src/internals/bptree/cursor.rs#L76">concread impl lincowcell</a></p> <h2 id="conclusion">Conclusion</h2> <p>Within async and sync programming, we can have cancellations or errors at any time - ensuring our applications are consistent in the case of errors which <em>will</em> happen, is challenging. By treating our internal APIs as a transactional interface, and applying database techniques we can create systems that are &quot;always consistent&quot;. It is possible to create these interfaces in a way that the Rust compiler can support us through it's type system to ensure we are using the correct transactional interfaces as we write our programs - helping us to move from just memory safety to broader application safety.</p> Results from the OpenSUSE 2021 Rust Survey Fri, 08 Oct 2021 00:00:00 +0000 Unknown https://fy.blackhats.net.au/blog/2021-10-08-results-from-the-opensuse-2021-rust-survey/ https://fy.blackhats.net.au/blog/2021-10-08-results-from-the-opensuse-2021-rust-survey/ <h1 id="results-from-the-opensuse-2021-rust-survey">Results from the OpenSUSE 2021 Rust Survey</h1> <p>From September the 8th to October the 7th, OpenSUSE has helped me host a survey on how developers are using Rust in their environments. As the maintainer of the Rust packages in SUSE and OpenSUSE it was important for me to get a better understanding of how people are using Rust so that we can make decisions that match how the community is working.</p> <p>First, to every single one of the 1360 people who responded to this survey, thank you! This exceeded my expectations and it means a lot to have had so many people take the time to help with this.</p> <p>All the data can be <a href="https://github.com/Firstyear/rust-survey/tree/main/2021">found here</a></p> <h2 id="what-did-you-want-to-answer">What did you want to answer?</h2> <p>I had assistance from a psychology researcher at a local university to construct the survey and her help guided the structure and many of the questions. An important element of this was that the questions provided shouldn't influence people into a certain answer, and that meant questions were built in a way to get a fair response that didn't lead people into a certain outcome or response pattern. As a result, it's likely that the reasons for the survey was not obvious to the participants.</p> <p>What we wanted to determine from this survey:</p> <ul> <li>How are developers installing rust toolchains so that we can attract them to OpenSUSE by reducing friction?</li> <li>In what ways are people using distribution rust packages in their environments (contrast to rustup)?</li> <li>Should our rust package include developer facing tools, or is it just another component of a build pipeline?</li> <li>When people create or distribute rust software, how are they managing their dependencies, and do we need to provide tools to assist?</li> <li>Based on the above, how can we make it easier for people to distribute rust software in packages as a distribution?</li> <li>How do developers manage security issues in rust libraries, and how can this be integrated to reduce packaging friction?</li> </ul> <h2 id="lets-get-to-the-data">Lets get to the data</h2> <p>As mentioned there were 1360 responses. Questions were broken into three broad categories.</p> <ul> <li>Attitude</li> <li>Developers</li> <li>Distributors</li> </ul> <h3 id="attitude">Attitude</h3> <p>This section was intended to be a gentle introduction to the survey, rather than answering any specific question. This section had 413 non-answers, which I will exclude for now.</p> <p>We asked three questions:</p> <ul> <li>Rust is important to my work or projects (1 disagree - 5 agree)</li> <li>Rust will become more important in my work or projects in the future.  (1 disagree - 5 agree)</li> <li>Rust will become more important to other developers and projects in the future (1 disagree - 5 agree)</li> </ul> <p><img src="/_static/rsurvey/1.png" alt="image" /></p> <p><img src="/_static/rsurvey/2.png" alt="image" /></p> <p><img src="/_static/rsurvey/3.png" alt="image" /></p> <p>From this there is strong support that rust is important to individuals today. It's likely this is biased as the survey was distributed mainly in rust communities, however, we still had 202 responses that were less than 3. Once we look at the future questions we see strong belief that rust will become more important. Again this is likely to be biased due to the communities the survey was distributed within, but we still see small numbers of people responding that rust will not be important to others or themself in the future.</p> <p>As this section was not intended to answer any questions, I have chosen not to use the responses of this section in other areas of the analysis.</p> <h3 id="developers">Developers</h3> <p>This section was designed to help answer the following questions:</p> <ul> <li>How are people installing rust toolchains so that we can attract them to OpenSUSE by reducing friction?</li> <li>In what ways are people using distribution rust packages in their environments (contrast to rustup)?</li> <li>Should our rust package include developer facing tools, or is it just another component of a build pipeline?</li> </ul> <p>We asked the following questions:</p> <ul> <li> <p>As a developer, I use Rust on the following platforms while programming.</p> </li> <li> <p>On your primary development platform, how did you install your Rust toolchain?</p> </li> <li></li> </ul> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>The following features or tools are important in my development environment (do not use 1 - use a lot 5) </span><span> </span><span>: - Integrated Development Environments with Language Features </span><span> (syntax highlight, errors, completion, type checking </span><span> - Debugging tools (lldb, gdb) </span><span> - Online Documentation (doc.rust-lang.org, docs.rs) </span><span> - Offline Documentation (local) </span><span> - Build Caching (sccache) </span></code></pre> <p>Generally we wanted to know what platforms people were using so that we could establish what people on linux were using <em>today</em> vs what people on other platforms were using, and then knowing what other platforms are doing we can make decisions about how to proceed.</p> <p><img src="/_static/rsurvey/4.png" alt="image" /></p> <p>There were 751 people who responded that they were a developer in this section. We can see Linux is the most popular platform used while programming, but for &quot;Linux only&quot; (derived by selecting responses that only chose Linux and no other platforms) this number is about equal to Mac and Windows. Given the prevalence of containers and other online linux environments it would make sense that developers access multiple platforms from their preferred OS, which is why there are many responses that selected multiple platforms for their work.</p> <p><img src="/_static/rsurvey/5.png" alt="image" /></p> <p>From the next question we see overwhelming support of rustup as the preferred method to install rust on most developer machines. As we did not ask &quot;why&quot; we can only speculate on the reasons for this decision.</p> <p><img src="/_static/rsurvey/6.png" alt="image" /></p> <p>When we isolate this to &quot;Linux only&quot;, we see a slight proportion increase in package manager installed rust environments, but there remains a strong tendancy for rustup to be the preferred method of installation.</p> <p>This may indicate that even within Linux distros with their package manager capabilities, and even with distributions try to provide rapid rust toolchain updates, that developers still prefer to use rust from rustup. Again, we can only speculate to why this is, but it already starts to highlight that distribution packaged rust is unlikely to be used as a developer facing tool.</p> <p><img src="/_static/rsurvey/7.png" alt="image" /></p> <p><img src="/_static/rsurvey/8.png" alt="image" /></p> <p><img src="/_static/rsurvey/9.png" alt="image" /></p> <p>Once we start to look at features of rust that developers rely on we see a very interesting distribution. I have not included all charts here. Some features are strongly used (IDE rls, online docs) where others seem to be more distributed in attitude (debuggers, offline docs, build caching). From the strongly supported features when we filter this by linux users using distribution packaged rust, we see a similar (but not as strong) trend for importance of IDE features. The other features like debuggers, offline docs and build caching all remain very distributed. This shows that tools like rls for IDE integration are very important, but with only a small number of developers using packaged rust as developers versus rustup it may not be an important area to support with limited packaging resources and time. It's very likely that developers who are on other distributions, mac or windows are more comfortable with a rustup based installation process.</p> <h3 id="distributors">Distributors</h3> <p>This section was designed to help answer the following questions:</p> <ul> <li>Should our rust package include developer facing tools, or is it just another component of a build pipeline?</li> <li>When people create or distribute rust software, how are they managing their dependencies, and do we need to provide tools to assist?</li> <li>Based on the above, how can we make it easier for people to distribute rust software in packages as a distribution?</li> <li>How do developers manage security issues in rust libraries, and how can this be integrated to reduce packaging friction?</li> </ul> <p>We asked the following questions:</p> <ul> <li>Which platforms (operating systems) do you target for Rust software</li> <li>How do you or your team/community build or provide Rust software for people to use?</li> <li>In your release process, how do you manage your Rust dependencies?</li> <li>In your ideal workflow, how would you prefer to manager your Rust dependencies?</li> <li>How do you manage security updates in your Rust dependencies?</li> </ul> <p><img src="/_static/rsurvey/10.png" alt="image" /></p> <p>Our first question here really shows the popularity of Linux as a target platform for running rust with 570 out of 618 responses indicating they target Linux as a platform.</p> <p><img src="/_static/rsurvey/11.png" alt="image" /></p> <p>Once we look at the distribution methods, both building projects to packages and using distribution packaged rust in containers fall well behind the use of rustup in containers and locally installed rust tools. However if we observe container packaged rust and packaged rust binaries (which likely use the distro rust toolchains) we have 205 uses of the rust package out of 1280 uses, where we see 59 out of 680 from developers. This does indicate a tendancy that the rust package in a distribution is more likely to be used in a build pipeline over developer use - but rustup still remains most popular. I would speculate that this is because developers want to recreate the same process on their development systems as their target systems which would likely involve rustup as the method to ensure the identical toolchains are installed.</p> <p>The next questions were focused on rust dependencies - as a staticly linked language, this changes the approach to how libraries can be managed. To answer how we as a distribution should support people in the way they want to manage libraries, we need to know how they use it today, and how they would ideally prefer to manage this in the future.</p> <p><img src="/_static/rsurvey/12.png" alt="image" /></p> <p><img src="/_static/rsurvey/13.png" alt="image" /></p> <p>In both the current process and ideal processes we see a large tendancy to online library use from crates.io, and in both cases vendoring (pre-downloading) comes in second place. Between the current process and ideal process, we see a small reduction in online library use to the other options. As a distribution, since we can not provide online access to crates, we can safely assume most online crates users would move to vendoring if they had to work offline for packaging as it's the most similar process available.</p> <p><img src="/_static/rsurvey/14.png" alt="image" /></p> <p><img src="/_static/rsurvey/15.png" alt="image" /></p> <p>We can also look at some other relationships here. People who provide packages still tend to ideally prefer online crates usage, with distribution libraries coming in second place here. There is still significant momentum for packagers to want to use vendoring or online dependencies though. When we look at ideal management strategies for container builds, we see distribution packages being much less popular, and online libraries still remaining at the top.</p> <p><img src="/_static/rsurvey/16.png" alt="image" /></p> <p>Finally, when we look at how developers are managing their security updates, we see a really healthy statistic that many people are using tools like cargo audit and cargo outdated to proactively update their dependencies. Very few people rely on distribution packages for their updates however. But it remains that we see 126 responses from users who aren't actively following security issues which again highlights a need for distributions who do provide rust packaged software to be proactive to detect issues that may exist.</p> <h2 id="outcomes">Outcomes</h2> <p>By now we have looked at a lot of the survey and the results, so it's time to answer our questions.</p> <ul> <li>How are people installing rust toolchains so that we can attract them to OpenSUSE by reducing friction?</li> </ul> <p>Developers are preferring the use of rustup over all other sources. Being what's used on linux and other platforms, we should consider packaging and distributing rustup to give options to users (who may wish to avoid the [curl | sh]{.title-ref} method.) I've already started the process to include this in OpenSUSE tumbleweed.</p> <ul> <li>In what ways are people using distribution rust packages in their environments (contrast to rustup)?</li> <li>Should our rust package include developer facing tools, or is it just another component of a build pipeline?</li> </ul> <p>Generally developers tend strongly to rustup for their toolchains, where distribution rust seems to be used more in build pipelines. As a result of the emphasis on online docs and rustup, we can likely remove offline documentation and rls from the distribution packages as they are either not being used or have very few users and is not worth the distribution support cost and maintainer time. We would likely be better to encourage users to use rustup for developer facing needs instead.</p> <p>To aid this argument, it appears that rls updates have been not functioning in OpenSUSE tumbleweed for a few weeks due to a packaging mistake, and no one has reported the issue - this means that the &quot;scream test&quot; failed. The lack of people noticing this again shows developer tools are not where our focus should be.</p> <ul> <li>When people create or distribute rust software, how are they managing their dependencies, and do we need to provide tools to assist?</li> <li>Based on the above, how can we make it easier for people to distribute rust software in packages as a distribution?</li> </ul> <p>Distributors prefer cargo and it's native tools, and this is likely an artifact of the tight knit tooling that exists in the rust community. Other options don't seem to have made a lot of headway, and even within distribution packaging where you may expect stronger desire for packaged libraries, we see a high level of support for cargo directly to manage rust dependencies. From this I think it shows that efforts to package rust crates have not been effective to attract developers who are currently used to a very different workflow.</p> <ul> <li>How do developers manage security issues in rust libraries, and how can this be integrated to reduce packaging friction?</li> </ul> <p>Here we see that many people are proactive in updating their libraries, but there still exists many who don't actively manage this. As a result, automating tools like cargo audit inside of build pipelines will likely help packagers, and also matches their existing and known tools. Given that many people will be performing frequent updates of their libraries or upstream releases, we'll need to also ensure that the process to update and commit updates to packages is either fully automated or at least has a minimal hands on contact as possible. When combined with the majority of developers and distributors prefering online crates for dependencies, encouraging people to secure these existing workflows will likely be a smaller step for them. Since rust is staticly linked, we can also target our security efforts at leaf (consuming) packages rather than the libraries themself.</p> <h2 id="closing">Closing</h2> <p>Again, thank you to everyone who answered the survey. It's now time for me to go and start to do some work based on this data!</p> Gnome 3 compare to MacOs Sun, 12 Sep 2021 00:00:00 +0000 Unknown https://fy.blackhats.net.au/blog/2021-09-12-gnome-3-compare-to-macos/ https://fy.blackhats.net.au/blog/2021-09-12-gnome-3-compare-to-macos/ <h1 id="gnome-3-compare-to-macos">Gnome 3 compare to MacOs</h1> <p>An assertion I have made in the past is that to me &quot;Gnome 3 feels like MacOs with rough edges&quot;. After some discussions with others, I'm finally going to write this up with examples.</p> <p>It's worth pointing out that in my opinion, Gnome 3 is probably still the best desktop experience on Linux today for a variety of reasons - it's just that for me, these rough edges really take away from that being a good experience for me.</p> <h2 id="high-level-structure-comparison">High Level Structure Comparison</h2> <p>Here's a pair of screenshots of MacOS 11.5.2 and Gnome 40.4. In both we have the settings menu open of the respective environment. Both are set to the resolution of 1680x1050, with the Mac using scaling from retina (2880x1800) to this size.</p> <p><img src="/_static/gnome_v_macos/gnome-settings-1.png" alt="image" /></p> <p><img src="/_static/gnome_v_macos/macos-settings-1.png" alt="image" /></p> <p>From this view, we can already make some observations. Both of these have a really similar structure which when we look at appears like this:</p> <p><img src="/_static/gnome_v_macos/skeleton.png" alt="image" /></p> <p>The skeleton overall looks really similar, if not identical. We have a top bar that provides a system tray and status and a system context in the top left, as well as application context.</p> <p>Now we can look at some of the details of each of the platforms at a high level from this skeleton.</p> <p>We can see on the Mac that the &quot;top menu bar&quot; takes 2.6% of our vertical screen real-estate. Our system context is provided by the small Apple logo in the top left that opens to a menu of various platform options.</p> <p>Next to that, we can see that our system preferences uses that top menu bar to provide our application context menus like edit, view, window and help. Further, on the right side of this we have a series of icons for our system - some of these from third party applications like nextcloud, and others coming from macos showing our backup status, keyboard, audio, battery, wifi time and more. This is using the space at the top of our screen really effectively, it doesn't feel wasted, and adds context to what we are doing.</p> <p>If we now look at Gnome we can see a different view. Our menu bar takes 3.5% of our vertical screen realestate, and the dark colour already feels like it is &quot;dominating&quot; visually. In that we have very little effective horizontal space use. The activities button (system context) takes us to our overview screen, and selecting the &quot;settings&quot; item which is our current application has no response or menu displayed.</p> <p>The system tray doesn't allow 3rd party applications, and the overview only shows our network and audio status and our clock (battery may be displayed on a laptop). To find more context about our system requires interaction with the single component at the top right, limiting our ability to interact with a specific element (network, audio etc) or understand our systems state quickly.</p> <p>Already we can start to see some differences here.</p> <ul> <li>UI elements in MacOS are smaller and consume less screen space.</li> <li>Large amounts of non-functional dead space in Gnome</li> <li>Elements are visually more apparently and able to be seen at a high level, where Gnome's require interaction to find details</li> </ul> <h2 id="system-preferences-vs-settings">System Preferences vs Settings</h2> <p>Let's compare the system preferences and Settings now. These are still similar, but not as close as our overall skeleton and this is where we start to see more about the different approaches to design in each.</p> <p>The MacOS system preferences has all of it's top level options displayed in a grid, with an easily accesible search function and forward and back navigation aides. This make it easy to find the relevant area that is required, and everything is immediately accessible and clear. Searching for items dims the application and begins to highlight elements that contain the relevant topic, helping to guide you to the location and establishing to the user where they can go in the future without the need to search. Inside any menu of the system preferences, search is always accesible and in the same consistent location of the application.</p> <p><img src="/_static/gnome_v_macos/macos-settings-search.png" alt="image" /></p> <p>When we look at Gnome, in the settings application we see that not all available settings are displayed - the gutter column on the left is a scrollable UI element, but with no scroll bars present, this could be missed by a user that the functionality is present. Items like &quot;Applications&quot; which have a &quot;&gt;&quot; present confusingly changes the gutter context to a list of applications rather than remaining at the top level when selected like all other items that don't have the &quot;&gt;&quot;. Breaking the users idea of consistency, when in these sub-gutters, the search icon is replaced with the &quot;back&quot; navigation icon, meaning you can not search when in a sub-gutter.</p> <p>Finally, even visually we can see that the settings is physically larger as a window, with much larger fonts and the title bar containing much more dead space. The search icon (when present) requires interaction before the search text area appears adding extra clicks and interactions to achieve the task.</p> <p>When we do search, the results are replaced into the gutter element. Screen lock here is actually in a sub-gutter menu for privacy, and not discoverable at the top level as an element. The use of nested gutters here adds confusion about where items are due to all the gutter content changes.</p> <p><img src="/_static/gnome_v_macos/gnome-settings-search.png" alt="image" /></p> <p>Again we are starting to see differences here:</p> <ul> <li>MacOS search uses greater visual feedback to help guide users to where they need to be</li> <li>Gnome hides many options in sub-menus, or with very few graphical guides which hinders discovery of items</li> <li>Again, the use of dead space in Gnome vs the greater use of space in MacOS</li> <li>Gnome requires more interactions to &quot;get around&quot; in general</li> <li>Gnome applications visually are larger and take up more space of the screen</li> <li>Gnome changes the UI and layout in subtle and inconsistent ways that rely on contextual knowledge of &quot;where&quot; you currently are in the application</li> </ul> <h2 id="context-menus">Context Menus</h2> <p>Lets have a look at some of the menus that exist in the system tray area now. For now I'll focus on audio, but these differences broadly apply to all of the various items here on MacOS and Gnome.</p> <p>On MacOS when we select our audio icon in the system tray, we are presented with a menu that contains the current volume, the current audio output device (including options for network streaming) and a link to the system preferencs control panel for further audio settings that may exist. We aren't overwhelmed with settings or choices, but we do have the ability to change our common options and shortcut links to get to the extended settings if needed.</p> <p><img src="/_static/gnome_v_macos/macos-audio-1.png" alt="image" /></p> <p>A common trick in MacOS though is holding the option key during interactions. Often this can display power-user or extended capabilities. When done on the audio menu, we are also able to then control our input device selection.</p> <p><img src="/_static/gnome_v_macos/macos-audio-2.png" alt="image" /></p> <p>On Gnome, in the system tray there is only a single element, that controls audio, power, network and more.</p> <p><img src="/_static/gnome_v_macos/gnome-audio-1.png" alt="image" /></p> <p>All we can do in this menu is control the volume - that's it. There are no links to direct audio settings, device management, and there are no &quot;hidden&quot; shortcuts (like option) that allows greater context or control.</p> <p>To summarise our differences:</p> <ul> <li>MacOS provides topic-specific system tray menus, with greater functionality and links to further settings</li> <li>Gnome has a combined menu, that is limited in functionality, and has only a generic link to settings</li> <li>Gnome lacks the ability to gain extended options for power-users to view extra settings or details</li> </ul> <h2 id="file-browser">File Browser</h2> <p>Finally lets look at the file browser. For fairness, I've changed Gnome's default layout to &quot;list&quot; to match my own usage in finder.</p> <p><img src="/_static/gnome_v_macos/macos-files-1.png" alt="image" /></p> <p>We can already see a number of useful elements here. We have the ability to &quot;tree&quot; folders through the &quot;&gt;&quot; icon, and rows of the browser alternate white/grey to help us visually identify lines horizontally. The rows are small and able to have (in this screenshot) 16 rows of content on the screen simultaneously. Finally, not shown here, but MacOS finder can use tabs for browsing different locations. And as before, we have our application context menu in the top bar with a large amount of actions available.</p> <p><img src="/_static/gnome_v_macos/gnome-files-1.png" alt="image" /></p> <p>Gnomes rows are all white with extremely faint grey lines to delineate, making it hard to horizontally track items if the window was expanded. The icons are larger, and there is no ability to tree the files and folders. We can only see ~10 rows on screen despite the similar size of the windows presented here. Finally, the extended options are hidden in the &quot;burger&quot; menu next to the application close.</p> <p>A theme should be apparent here:</p> <ul> <li>Both MacOS and Gnome share a very similar skeleton of how this application is laid out</li> <li>MacOS makes better use of visual elements to help your eye track across spaces to make connections</li> <li>Gnome has a lot of dead space still and larger text and icons which takes greater amounts of screen space</li> <li>Due to the application context and other higher level items, MacOS is &quot;faster&quot; to get to where you need to go</li> </ul> <h2 id="keyboard-shortcuts">Keyboard Shortcuts</h2> <p>Keyboard shortcuts are something that aide powerusers to achieve tasks quicker, but the challenge is often <em>finding</em> what shortcuts exist to use them. Lets look at how MacOS and Gnome solve this.</p> <p><img src="/_static/gnome_v_macos/macos-shortcut-1.png" alt="image" /></p> <p>Here in MacOS, anytime we open a menu, we can see the shortcut listed next to the menu item that is present, including disabled items (that are dimmed). Each shortcut's symbols match the symbols of the keyboard allowing these to be cross-language and accessible. And since we are in a menu, we remain in the context of our Application and able to then immediately use the menu or shortcut.</p> <p>In fact, even if we select the help menu and search a new topic, rather than take us away from menu's, MacOS opens the menu and points us to where we are trying to go, allowing us to find the action we want <em>and</em> learn it's shortcut!</p> <p><img src="/_static/gnome_v_macos/macos-shortcut-2.png" alt="image" /></p> <p>This is great, because it means in the process of getting help, we are shown how to perform the action for future interactions. Because of the nature of MacOS human interface guidelines this pattern exists for <em>all</em> applications on the platform, including third party ones helping to improve accessibility of these features.</p> <p>Gnome however takes a really different approach. Keyboard shortcuts are listed as a menu item from our burger menu.</p> <p><img src="/_static/gnome_v_macos/gnome-shortcut-1.png" alt="image" /></p> <p>When we select it, our applications context is taken away and replaced with a dictionary of keyboard shortcuts, spread over three pages.</p> <p><img src="/_static/gnome_v_macos/gnome-shortcut-2.png" alt="image" /></p> <p>I think the use of the keyboard icons here is excellent, but because we are now in a dictionary of shortcuts, it's hard to find what we want to use, and we &quot;taken away&quot; from the context of the actions we are trying to perform in our application. Again, we have to perform more interactions to find the information that we are looking for in our applications, and we aren't able to easily link the action to the shortcut in this style of presentation. We can't transfer our knowledge of the &quot;menus&quot; into a shortcut that we can use without going through a reference manual.</p> <p>Another issue here is this becomes the responsibility of each application to create these references and provide them, rather than being an automatically inherited feature through the adherence to human interface guidelines.</p> <h2 id="conclusion">Conclusion</h2> <p>Honestly, I could probably keep making these comparisons all day. Gnome 3 and MacOS really do feel very similar to me. From style of keyboard shortcuts, layout of the UI, the structure of it's applications and even it's approach to windowing feels identical to MacOS. However while it looks similar on a surface level, there are many rough edges, excess interactions, poor use of screen space and visual elements.</p> <p>MacOS certainly has it's flaws, and makes it's mistakes. But from a ease of use perspective, it tries to get out of the way and show you how to use the computer for yourself. MacOS takes a back seat to the usage of the computer.</p> <p>Gnome however feels like it wants to be front and centre. It needs you to know all the time &quot;you're using Gnome!&quot;. It takes you on a small adventure tour to complete simple actions or to discover new things. It even feels like Gnome has tried to reduce &quot;complexity&quot; so much that they have thrown away many rich features and interactions that could make a computer easier to use and interact with.</p> <p>So for me, this is why I feel that Gnome is like MacOS with rough edges. There are many small, subtle and frustrating user interactions like this all through out the Gnome 3 experience that just aren't present in MacOS.</p> StartTLS in LDAP Thu, 12 Aug 2021 00:00:00 +0000 Unknown https://fy.blackhats.net.au/blog/2021-08-12-starttls-in-ldap/ https://fy.blackhats.net.au/blog/2021-08-12-starttls-in-ldap/ <h1 id="starttls-in-ldap">StartTLS in LDAP</h1> <p>LDAP as a protocol is a binary protocol which uses ASN.1 BER encoded structures to communicate between a client and server, to query directory information (ie users, groups, locations, etc).</p> <p>When this was created there was little consideration to security with regard to person-in-the-middle attacks (aka mitm: meddler in the middle, interception). As LDAP has become used not just as a directory service for accessing information, but now as an authentication and authorisation system it's important that the content of these communications is secure from tampering or observation.</p> <p>There have been a number of introduced methods to try and assist with this situation. These are:</p> <ul> <li>StartTLS</li> <li>SASL with encryption layers</li> <li>LDAPS (LDAP over TLS)</li> </ul> <p>Other protocols of a similar age also have used StartTLS such as SMTP and IMAP. However recent <a href="https://nostarttls.secvuln.info/">research</a> has (again) shown issues with correct StartTLS handling, and recommends using SMTPS or IMAPS.</p> <p>Today the same is true of LDAP - the only secure method of communication to an LDAP server is LDAPS. In this blog, I'll be exploring the issues that exist with StartTLS (I will not cover SASL or GSSAPI).</p> <h2 id="how-does-starttls-work">How does StartTLS work?</h2> <p>StartTLS works by starting a plaintext (unencrypted) connection to the LDAP server, and then by upgrading that connection to begin TLS within the existing connection.</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>┌───────────┐ ┌───────────┐ </span><span>│ │ │ │ </span><span>│ │─────────open tcp 389──────▶│ │ </span><span>│ │◀────────────ok─────────────│ │ </span><span>│ │ │ │ </span><span>│ │ │ │ </span><span>│ │────────ldap starttls──────▶│ │ </span><span>│ │◀──────────success──────────│ │ </span><span>│ │ │ │ </span><span>│ Client │ │ Server │ </span><span>│ │──────tls client hello─────▶│ │ </span><span>│ │◀─────tls server hello──────│ │ </span><span>│ │────────tls key xchg───────▶│ │ </span><span>│ │◀────────tls finish─────────│ │ </span><span>│ │ │ │ </span><span>│ │──────TLS(ldap bind)───────▶│ │ </span><span>│ │ │ │ </span><span>│ │ │ │ </span><span>└───────────┘ └───────────┘ </span></code></pre> <p>As we can see in LDAP StartTLS we establish a valid plaintext tcp connection, and then we send and LDAP message containing a StartTLS extended operation. If successful, we begin a TLS handshake over the connection, and when complete, our traffic is now encrypted.</p> <p>This is contrast to LDAPS where TLS must be successfully established before the first LDAP message is exchanged.</p> <p>It's a good time to note that this is inefficent as it takes an extra round-trip to establish StartTLS like this contrast to LDAPS which increases latency for all communications. LDAP clients tend to open and close many connections, so this adds up quickly.</p> <h2 id="security-issues">Security Issues</h2> <h3 id="client-misconfiguration">Client Misconfiguration</h3> <p>LDAP servers at the start of a connection will only accept two LDAP messages. Bind (authenticate) and StartTLS. Since StartTLS starts with a plaintext connection, if a client is misconfigured it is trivial for it to operate without StartTLS.</p> <p>For example, consider the following commands.</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span># ldapwhoami -H ldap://172.17.0.3:389 -x -D &#39;cn=Directory Manager&#39; -W </span><span>Enter LDAP Password: </span><span>dn: cn=directory manager </span><span># ldapwhoami -H ldap://172.17.0.3:389 -x -Z -D &#39;cn=Directory Manager&#39; -W </span><span>Enter LDAP Password: </span><span>dn: cn=directory manager </span></code></pre> <p>Notice that in both, the command succeeds and we authenticate. However, only in the second command are we using StartTLS. This means we trivially leaked our password. Forcing LDAPS to be the only protocol prevents this as every byte of the connection is always encrypted.</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span># ldapwhoami -H ldaps://172.17.0.3:636 -x -D &#39;cn=Directory Manager&#39; -W </span><span>Enter LDAP Password: </span><span>dn: cn=directory manager </span></code></pre> <p>Simply put this means that if you forget to add the command line flag for StartTLS, forget the checkbox in an admin console, or any other kind of possible human error (which happen!), then LDAP will silently continue without enforcing that StartTLS is present.</p> <p>For a system to be secure we must prevent human error from being a factor by removing elements of risk in our systems.</p> <h3 id="minssf">MinSSF</h3> <p>A response to the above is to enforce MinSSF, or &quot;Minimum Security Strength Factor&quot;. This is an option on both OpenLDAP and 389-ds and is related to the integration of SASL. It represents that the bind method used must have &quot;X number of bits&quot; of security (however X is very arbitrary and not really representative of true security).</p> <p>In the context of StartTLS or TLS, the provided SSF becomes the number of bits in the symmetric encryption used in the connection. Generally this is 128 due to the use of AES128.</p> <p>Let us assume we have configured MinSSF=128 and we attempt to bind to our server.</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>┌───────────┐ ┌───────────┐ </span><span>│ │ │ │ </span><span>│ │─────────open tcp 389──────▶│ │ </span><span>│ │◀────────────ok─────────────│ │ </span><span>│ Client │ │ Server │ </span><span>│ │ │ │ </span><span>│ │──────────ldap bind────────▶│ │ </span><span>│ │◀───────error - minssf──────│ │ </span><span>│ │ │ │ </span><span>└───────────┘ └───────────┘ </span></code></pre> <p>The issue here is the minssf isn't enforced until the bind message is sent. If we look at the LDAP rfc we see:</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>BindRequest ::= [APPLICATION 0] SEQUENCE { </span><span> version INTEGER (1 .. 127), </span><span> name LDAPDN, </span><span> authentication AuthenticationChoice } </span><span> </span><span>AuthenticationChoice ::= CHOICE { </span><span> simple [0] OCTET STRING, </span><span> -- 1 and 2 reserved </span><span> sasl [3] SaslCredentials, </span><span> ... } </span><span> </span><span>SaslCredentials ::= SEQUENCE { </span><span> mechanism LDAPString, </span><span> credentials OCTET STRING OPTIONAL } </span></code></pre> <p>Which means that in a simple bind (password) in the very first message we send our plaintext password. MinSSF only tells us <em>after</em> we already made the mistake, so this is not a suitable defence.</p> <h3 id="starttls-can-be-disregarded">StartTLS can be disregarded</h3> <p>An interesting aspect of how StartTLS works with LDAP is that it's possible to prevent it from being installed successfully. If we look at the <a href="https://datatracker.ietf.org/doc/html/rfc4511#section-4.14.2">RFC</a>:</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>If the server is otherwise unwilling or unable to perform this </span><span>operation, the server is to return an appropriate result code </span><span>indicating the nature of the problem. For example, if the TLS </span><span>subsystem is not presently available, the server may indicate this by </span><span>returning with the resultCode set to unavailable. In cases where a </span><span>non-success result code is returned, the LDAP session is left without </span><span>a TLS layer. </span></code></pre> <p>What this means is it is up to the client and how they respond to this error to enforce a correct behaviour. An example of a client that disregards this error may proceed such as:</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>┌───────────┐ ┌───────────┐ </span><span>│ │ │ │ </span><span>│ │─────────open tcp 389──────▶│ │ </span><span>│ │◀────────────ok─────────────│ │ </span><span>│ │ │ │ </span><span>│ Client │ │ Server │ </span><span>│ │────────ldap starttls──────▶│ │ </span><span>│ │◀───────starttls error──────│ │ </span><span>│ │ │ │ </span><span>│ │─────────ldap bind─────────▶│ │ </span><span>│ │ │ │ </span><span>└───────────┘ └───────────┘ </span></code></pre> <p>In this example, the ldap bind proceeds even though TLS is not active, again leaking our password in plaintext. A classic example of this is OpenLDAP's own cli tools which in almost all examples of StartTLS online use the option '-Z' to enable this.</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span># ldapwhoami -Z -H ldap://127.0.0.1:12345 -D &#39;cn=Directory Manager&#39; -w password </span><span>ldap_start_tls: Protocol error (2) </span><span>dn: cn=Directory Manager </span></code></pre> <p>The quirk is that '-Z' here only means to <em>try</em> StartTLS. If you want to fail when it's not available you need '-ZZ'. This is a pretty easy mistake for any administrator to make when typing a command. There is no way to configure in ldap.conf that you always want StartTLS enforced either leaving it again to human error. Given the primary users of the ldap cli are directory admins, this makes it a high value credential open to potential human input error.</p> <p>Within client applications a similar risk exists that the developers need to correctly enforce this behaviour. Thankfully for us, the all client applications that I tested handle this correctly:</p> <ul> <li>SSSD</li> <li>nslcd</li> <li>ldapvi</li> <li>python-ldap</li> </ul> <p>However, I am sure there are many others that should be tested to ensure that they correctly handle errors during StartTLS.</p> <h3 id="referral-injection">Referral Injection</h3> <p>Referral's are a feature of LDAP that allow responses to include extra locations where a client may look for the data they requested, or to extend the data they requested. Due to the design of LDAP and it's response codes, referrals are valid in all response messages.</p> <p>LDAP StartTLS does allow a referral as a valid response for the client to then follow - this may be due to the requested server being undermaintenance or similar.</p> <p>Depending on the client implementation, this may allow an mitm to proceed. There are two possible scenarioes.</p> <p>Assuming the client <em>does</em> do certificate validation, but is poorly coded, the following may occur:</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>┌───────────┐ ┌───────────┐ </span><span>│ │ │ │ </span><span>│ │─────────open tcp 389──────▶│ │ </span><span>│ │◀────────────ok─────────────│ │ </span><span>│ │ │ Server │ </span><span>│ │ │ │ </span><span>│ │────────ldap starttls──────▶│ │ </span><span>│ │◀──────────referral─────────│ │ </span><span>│ │ │ │ </span><span>│ │ └───────────┘ </span><span>│ Client │ </span><span>│ │ ┌───────────┐ </span><span>│ │─────────ldap bind─────────▶│ │ </span><span>│ │ │ │ </span><span>│ │ │ │ </span><span>│ │ │ Malicious │ </span><span>│ │ │ Server │ </span><span>│ │ │ │ </span><span>│ │ │ │ </span><span>│ │ │ │ </span><span>└───────────┘ └───────────┘ </span></code></pre> <p>In this example our server sent a referral as a response to the StartTLS extended operation, which the client then followed - however the client did <em>not</em> attempt to install StartTLS again when contacting the malicious server. This would allow a bypass of certification validation by simply never letting TLS begin at all. Thankfully the clients I tested did not exhibt this behaviour, but it is possible.</p> <p>If the client has configured certificate validation to never (tls_reqcert = never, which is a surprisingly common setting ...) then the following is possible.</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>┌───────────┐ ┌───────────┐ </span><span>│ │ │ │ </span><span>│ │─────────open tcp 389──────▶│ │ </span><span>│ │◀────────────ok─────────────│ │ </span><span>│ │ │ Server │ </span><span>│ │ │ │ </span><span>│ │────────ldap starttls──────▶│ │ </span><span>│ │◀──────────referral─────────│ │ </span><span>│ │ │ │ </span><span>│ │ └───────────┘ </span><span>│ Client │ </span><span>│ │ ┌───────────┐ </span><span>│ │────────ldap starttls──────▶│ │ </span><span>│ │◀──────────success──────────│ │ </span><span>│ │ │ │ </span><span>│ │◀──────TLS installed───────▶│ Malicious │ </span><span>│ │ │ Server │ </span><span>│ │───────TLS(ldap bind)──────▶│ │ </span><span>│ │ │ │ </span><span>│ │ │ │ </span><span>└───────────┘ └───────────┘ </span></code></pre> <p>In this example the client follows the referral and then attempts to install StartTLS again. The malicious server may present any certificate it wishes and can then intercept traffic.</p> <p>In my testing I found that this affected both SSSD and nslcd, however both of these when redirected to the malicous server would attempt to install StartTLS over an existing StartTLS channel, which caused the server to return an error condition. Potentially a modified malicious server in this case would be able to install two layers of TLS, or a response that would successfully trick these clients to divulging further information. I have not yet spent time to research this further.</p> <h2 id="conclusion">Conclusion</h2> <p>While not as significant as the results found on &quot;No Start TLS&quot;, LDAP still is potentially exposed to risks related to StartTLS usage. To mitigate these LDAP server providers should disable plaintext LDAP ports and exclusively use LDAPS, with tls_reqcert set to &quot;demand&quot;.</p> Getting started with Yew Sun, 20 Jun 2021 00:00:00 +0000 Unknown https://fy.blackhats.net.au/blog/2021-06-20-getting-started-with-yew/ https://fy.blackhats.net.au/blog/2021-06-20-getting-started-with-yew/ <h1 id="getting-started-with-yew">Getting started with Yew</h1> <blockquote> <p><strong>NOTE</strong> This post is really out dated now, there are easier ways to start. See the <a href="https://yew.rs/">yew official docs</a> as this process has gotten much easier!</p> </blockquote> <p>Yew is a really nice framework for writing single-page-applications in Rust, that is then compiled to wasm for running in the browser. For me it has helped make web development much more accessible to me, but getting started with it isn't always straight forward.</p> <p>This is the bare-minimum to get a &quot;hello world&quot; in your browser - from there you can build on that foundation to make many more interesting applications.</p> <h2 id="dependencies">Dependencies</h2> <h3 id="macos">MacOS</h3> <ul> <li>Ensure that you have rust, which you can setup with <a href="https://rustup.rs/">RustUp</a>.</li> <li>Ensure that you have brew, which you can install from the <a href="https://brew.sh/">Homebrew Project</a>. This is used to install other tools.</li> <li>Install wasm-pack. wasm-pack is what drives the rust to wasm build process.</li> </ul> <pre data-lang="bash" style="background-color:#2b303b;color:#c0c5ce;" class="language-bash "><code class="language-bash" data-lang="bash"><span> </span><span style="color:#bf616a;">cargo</span><span> install wasm-pack </span></code></pre> <ul> <li>Install npm and rollup. npm is needed to install rollup, and rollup is what takes our wasm and javacript and bundles them together for our browser.</li> </ul> <pre data-lang="bash" style="background-color:#2b303b;color:#c0c5ce;" class="language-bash "><code class="language-bash" data-lang="bash"><span> </span><span style="color:#bf616a;">brew</span><span> install npm </span><span> </span><span style="color:#bf616a;">npm</span><span> install</span><span style="color:#bf616a;"> --global</span><span> rollup </span></code></pre> <ul> <li>Install miniserve for hosting our website locally during development.</li> </ul> <pre data-lang="bash" style="background-color:#2b303b;color:#c0c5ce;" class="language-bash "><code class="language-bash" data-lang="bash"><span> </span><span style="color:#bf616a;">brew</span><span> install miniserve </span></code></pre> <h2 id="a-new-project">A new project</h2> <p>We can now create a new rust project. Note we use --lib to indicate that it's a library, not an executable.</p> <pre data-lang="bash" style="background-color:#2b303b;color:#c0c5ce;" class="language-bash "><code class="language-bash" data-lang="bash"><span> </span><span style="color:#bf616a;">cargo</span><span> new</span><span style="color:#bf616a;"> --lib</span><span> yewdemo </span></code></pre> <p>To start with we'll need some boilerplate and helpers to get ourselves started.</p> <p>[index.html]{.title-ref} - our default page that will load our wasm to run. This is our &quot;entrypoint&quot; into the site that starts everything else off. In this case it loads our bundled javascript.</p> <pre data-lang="html" style="background-color:#2b303b;color:#c0c5ce;" class="language-html "><code class="language-html" data-lang="html"><span> &lt;!</span><span style="color:#b48ead;">DOCTYPE </span><span style="color:#d08770;">html</span><span>&gt; </span><span> &lt;</span><span style="color:#bf616a;">html</span><span>&gt; </span><span> &lt;</span><span style="color:#bf616a;">head</span><span>&gt; </span><span> &lt;</span><span style="color:#bf616a;">meta </span><span style="color:#d08770;">charset</span><span>=&quot;</span><span style="color:#a3be8c;">utf-8</span><span>&quot;&gt; </span><span> &lt;</span><span style="color:#bf616a;">title</span><span>&gt;PROJECTNAME&lt;/</span><span style="color:#bf616a;">title</span><span>&gt; </span><span> &lt;</span><span style="color:#bf616a;">script </span><span style="color:#d08770;">src</span><span>=&quot;</span><span style="color:#a3be8c;">/pkg/bundle.js</span><span>&quot; </span><span style="color:#d08770;">defer</span><span>&gt;&lt;/</span><span style="color:#bf616a;">script</span><span>&gt; </span><span> &lt;/</span><span style="color:#bf616a;">head</span><span>&gt; </span><span> &lt;</span><span style="color:#bf616a;">body</span><span>&gt; </span><span> &lt;/</span><span style="color:#bf616a;">body</span><span>&gt; </span><span> &lt;/</span><span style="color:#bf616a;">html</span><span>&gt; </span></code></pre> <p>[main.js]{.title-ref} - this is our javascript entrypoint that we'll be using. Remember to change PROJECTNAME to your crate name (ie yewdemo). This will be combined with our wasm to create the bundle.js file.</p> <pre data-lang="js" style="background-color:#2b303b;color:#c0c5ce;" class="language-js "><code class="language-js" data-lang="js"><span> </span><span style="color:#b48ead;">import </span><span style="color:#bf616a;">init</span><span>, { </span><span style="color:#bf616a;">run_app </span><span>} </span><span style="color:#b48ead;">from </span><span>&#39;</span><span style="color:#a3be8c;">./pkg/PROJECTNAME.js</span><span>&#39;; </span><span> </span><span style="color:#b48ead;">async function </span><span style="color:#8fa1b3;">main</span><span>() { </span><span> </span><span style="color:#b48ead;">await </span><span style="color:#8fa1b3;">init</span><span>(&#39;</span><span style="color:#a3be8c;">/pkg/PROJECTNAME_bg.wasm</span><span>&#39;); </span><span> </span><span style="color:#8fa1b3;">run_app</span><span>(); </span><span> } </span><span> </span><span style="color:#8fa1b3;">main</span><span>() </span></code></pre> <p>[Cargo.toml]{.title-ref} - we need to extend Cargo.toml with some dependencies and settings that allows wasm to build and our framework dependencies.</p> <pre data-lang="toml" style="background-color:#2b303b;color:#c0c5ce;" class="language-toml "><code class="language-toml" data-lang="toml"><span> [lib] </span><span> </span><span style="color:#bf616a;">crate-type </span><span>= [&quot;</span><span style="color:#a3be8c;">cdylib</span><span>&quot;] </span><span> </span><span> [dependencies] </span><span> </span><span style="color:#bf616a;">wasm-bindgen </span><span>= &quot;</span><span style="color:#a3be8c;">^0.2</span><span>&quot; </span><span> </span><span style="color:#bf616a;">yew </span><span>= &quot;</span><span style="color:#a3be8c;">0.18</span><span>&quot; </span></code></pre> <p>[build_wasm.sh]{.title-ref} - create this file to help us build our project. Remember to call [chmod +x build_wasm.sh]{.title-ref} so that you can execute it later.</p> <pre data-lang="bash" style="background-color:#2b303b;color:#c0c5ce;" class="language-bash "><code class="language-bash" data-lang="bash"><span> </span><span style="color:#65737e;">#!/bin/sh </span><span> </span><span style="color:#bf616a;">wasm-pack</span><span> build</span><span style="color:#bf616a;"> --target</span><span> web &amp;&amp; \ </span><span> </span><span style="color:#bf616a;">rollup</span><span> ./main.js</span><span style="color:#bf616a;"> --format</span><span> iife</span><span style="color:#bf616a;"> --file</span><span> ./pkg/bundle.js </span></code></pre> <p>[src/lib.rs]{.title-ref} - this is a template of a minimal start point for yew. This has all the stubs in place for a minimal &quot;hello world&quot; website.</p> <pre data-lang="rust" style="background-color:#2b303b;color:#c0c5ce;" class="language-rust "><code class="language-rust" data-lang="rust"><span> </span><span style="color:#b48ead;">use </span><span>wasm_bindgen::prelude::*; </span><span> </span><span style="color:#b48ead;">use </span><span>yew::prelude::*; </span><span> </span><span style="color:#b48ead;">use </span><span>yew::services::ConsoleService; </span><span> </span><span> </span><span style="color:#b48ead;">pub struct </span><span>App { </span><span> </span><span style="color:#bf616a;">link</span><span>: ComponentLink&lt;</span><span style="color:#b48ead;">Self</span><span>&gt;, </span><span> } </span><span> </span><span> </span><span style="color:#b48ead;">impl </span><span>Component </span><span style="color:#b48ead;">for </span><span>App { </span><span> </span><span style="color:#b48ead;">type </span><span>Message = App; </span><span> </span><span style="color:#b48ead;">type </span><span>Properties = (); </span><span> </span><span> </span><span style="color:#65737e;">// This is called when our App is initially created. </span><span> </span><span style="color:#b48ead;">fn </span><span style="color:#8fa1b3;">create</span><span>(_: </span><span style="color:#b48ead;">Self::</span><span>Properties, </span><span style="color:#bf616a;">link</span><span>: ComponentLink&lt;</span><span style="color:#b48ead;">Self</span><span>&gt;) -&gt; </span><span style="color:#b48ead;">Self </span><span>{ </span><span> App { </span><span> link, </span><span> } </span><span> } </span><span> </span><span> </span><span style="color:#b48ead;">fn </span><span style="color:#8fa1b3;">change</span><span>(&amp;</span><span style="color:#b48ead;">mut </span><span style="color:#bf616a;">self</span><span>, _: </span><span style="color:#b48ead;">Self::</span><span>Properties) -&gt; ShouldRender { </span><span> </span><span style="color:#d08770;">false </span><span> } </span><span> </span><span> </span><span style="color:#65737e;">// Called during event callbacks initiated by events (user or browser) </span><span> </span><span style="color:#b48ead;">fn </span><span style="color:#8fa1b3;">update</span><span>(&amp;</span><span style="color:#b48ead;">mut </span><span style="color:#bf616a;">self</span><span>, </span><span style="color:#bf616a;">msg</span><span>: </span><span style="color:#b48ead;">Self::</span><span>Message) -&gt; ShouldRender { </span><span> </span><span style="color:#d08770;">false </span><span> } </span><span> </span><span> </span><span style="color:#65737e;">// Render our content to the page, emitting Html that will be loaded into our </span><span> </span><span style="color:#65737e;">// index.html&#39;s &lt;body&gt; </span><span> </span><span style="color:#b48ead;">fn </span><span style="color:#8fa1b3;">view</span><span>(&amp;</span><span style="color:#bf616a;">self</span><span>) -&gt; Html { </span><span> ConsoleService::log(&quot;</span><span style="color:#a3be8c;">Hello World!</span><span>&quot;); </span><span> html! { </span><span> &lt;div&gt; </span><span> &lt;h2&gt;{ &quot;</span><span style="color:#a3be8c;">Hello World</span><span>&quot; }&lt;/h2&gt; </span><span> &lt;/div&gt; </span><span> } </span><span> } </span><span> } </span><span> </span><span> </span><span style="color:#65737e;">// This is the entry point that main.js calls into. </span><span> #[</span><span style="color:#bf616a;">wasm_bindgen</span><span>] </span><span> </span><span style="color:#b48ead;">pub fn </span><span style="color:#8fa1b3;">run_app</span><span>() -&gt; Result&lt;(), JsValue&gt; { </span><span> yew::start_app::&lt;App&gt;(); </span><span> Ok(()) </span><span> } </span></code></pre> <h2 id="building-your-hello-world">Building your Hello World</h2> <p>Now you can build your project with:</p> <pre data-lang="bash" style="background-color:#2b303b;color:#c0c5ce;" class="language-bash "><code class="language-bash" data-lang="bash"><span> </span><span style="color:#bf616a;">./build_wasm.sh </span></code></pre> <p>And if you want to see it on your machine in your browser:</p> <pre data-lang="bash" style="background-color:#2b303b;color:#c0c5ce;" class="language-bash "><code class="language-bash" data-lang="bash"><span> </span><span style="color:#bf616a;">miniserve -v --index</span><span> index.html . </span></code></pre> <p>Navigate to <a href="http://127.0.0.1:8080">http://127.0.0.1:8080</a> to see your Hello World!</p> <h2 id="further-resources">Further Resources</h2> <ul> <li><a href="https://yew.rs/">yew guide</a></li> <li><a href="https://docs.rs/yew/0.18.0/yew/">yew api documentation</a></li> <li><a href="https://github.com/yewstack/yew/tree/master/examples">yew example projects</a></li> <li><a href="https://rustwasm.github.io/wasm-bindgen/introduction.html">wasm-bindgen book</a></li> </ul> <h2 id="troubleshooting">Troubleshooting</h2> <p>I made all the following mistakes while writing this blog 😅</p> <h3 id="build-wasm-sh-permission-denied">build_wasm.sh - permission denied</h3> <pre data-lang="bash" style="background-color:#2b303b;color:#c0c5ce;" class="language-bash "><code class="language-bash" data-lang="bash"><span> </span><span style="color:#bf616a;">./build_wasm.sh </span><span> </span><span style="color:#bf616a;">zsh:</span><span> permission denied: ./build_wasm.sh </span></code></pre> <p>You need to run &quot;chmod +x build_wasm.sh&quot; so that you can execute this. Permission denied means that the executable bits are missing from the file.</p> <h3 id="building-could-not-resolve">building - 'Could not resolve'</h3> <pre data-lang="bash" style="background-color:#2b303b;color:#c0c5ce;" class="language-bash "><code class="language-bash" data-lang="bash"><span> </span><span style="color:#bf616a;">./main.js</span><span> → ./pkg/bundle.js... </span><span> </span><span style="color:#bf616a;">[!]</span><span> Error: Could not resolve &#39;</span><span style="color:#a3be8c;">./pkg/PROJECTNAME.js</span><span>&#39; from main.js </span><span> </span><span style="color:#bf616a;">Error:</span><span> Could not resolve &#39;</span><span style="color:#a3be8c;">./pkg/PROJECTNAME.js</span><span>&#39; from main.js </span></code></pre> <p>This error means you need to edit main.js so that PROJECTNAME matches your crate name.</p> <h3 id="blank-page-in-browser">Blank Page in Browser</h3> <p>When you first load your page it may be blank. You can check if a file is missing or incorrectly named by right clicking the page, select 'inspect', and in the inspector go to the 'network' tab.</p> <p>From there refresh your page, and see if any files 404. If they do you may need to rename them or there is an error in yoru main.js. A common one is:</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span> PROJECTNAME.wasm: 404 </span></code></pre> <p>This is because in main.js you may have changed the await init line, and removed the suffix [_bg]{.title-ref}.</p> <pre data-lang="js" style="background-color:#2b303b;color:#c0c5ce;" class="language-js "><code class="language-js" data-lang="js"><span> # </span><span style="color:#bf616a;">Incorrect </span><span> </span><span style="color:#b48ead;">await </span><span style="color:#8fa1b3;">init</span><span>(&#39;</span><span style="color:#a3be8c;">/pkg/PROJECTNAME.wasm</span><span>&#39;); </span><span> # </span><span style="color:#bf616a;">Correct </span><span> </span><span style="color:#b48ead;">await </span><span style="color:#8fa1b3;">init</span><span>(&#39;</span><span style="color:#a3be8c;">/pkg/PROJECTNAME_bg.wasm</span><span>&#39;); </span></code></pre> Compiler Bootstrapping - Can We Trust Rust? Wed, 12 May 2021 00:00:00 +0000 Unknown https://fy.blackhats.net.au/blog/2021-05-12-compiler-bootstrapping-can-we-trust-rust/ https://fy.blackhats.net.au/blog/2021-05-12-compiler-bootstrapping-can-we-trust-rust/ <h1 id="compiler-bootstrapping-can-we-trust-rust">Compiler Bootstrapping - Can We Trust Rust?</h1> <p>Recently I have been doing a lot of work for SUSE with how we package the Rust compiler. This process has been really interesting and challenging, but like anything it's certainly provided a lot of time for thought while <a href="https://xkcd.com/303/">waiting</a> for my packages to build.</p> <p>The Rust package in OpenSUSE has two methods of building the compiler internally in it's spec file.</p> <ul> <li> <ol> <li>Use our previously packaged version of rustc from packages</li> </ol> </li> <li> <ol start="2"> <li>Bootstrap using the signed and prebuilt binaries provided by the rust project</li> </ol> </li> </ul> <h2 id="bootstrapping">Bootstrapping</h2> <p>There are many advocates of bootstrapping and then self sustaining a chain of compilers within a distribution. The roots of this come from Ken Thompsons Turing Award speech known as <a href="https://www.ece.cmu.edu/~ganger/712.fall02/papers/p761-thompson.pdf">Reflections on trusting trust</a> . This details the process in which a compiler can be backdoored, to produce future backdoored compilers. This has been replicated by Manish G. detailed in their <a href="https://manishearth.github.io/blog/2016/12/02/reflections-on-rusting-trust/">blog, Reflections on Rusting Trust</a> where they successfully create a self-hosting backdoored rust compiler.</p> <p>The process can be visualised as:</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>┌──────────────┐ ┌──────────────┐ </span><span>│ Backdoored │ │ Trusted │ </span><span>│ Sources │──────┐ │ Sources │──────┐ </span><span>│ │ │ │ │ │ </span><span>└──────────────┘ │ └──────────────┘ │ </span><span> │ │ </span><span>┌──────────────┐ │ ┌──────────────┐ │ ┌──────────────┐ </span><span>│ Trusted │ ▼ │ Backdoored │ ▼ │ Backdoored │ </span><span>│ Interpreter │──Produces───▶│ Binary ├──Produces──▶│ Binary │ </span><span>│ │ │ │ │ │ </span><span>└──────────────┘ └──────────────┘ └──────────────┘ </span></code></pre> <p>We can see that in this attack, even with a set of trusted compiler sources, we can continue to produce a chain of backdoored binaries.</p> <p>This has led to many people, and even groups such as <a href="https://www.bootstrappable.org/">Bootstrappable</a> promoting work to be able to produce trusted chains from trusted sources, so that we can assert a level of trust in our produced compiler binaries.</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>┌──────────────┐ ┌──────────────┐ </span><span>│ Trusted │ │ Trusted │ </span><span>│ Sources │──────┐ │ Sources │──────┐ </span><span>│ │ │ │ │ │ </span><span>└──────────────┘ │ └──────────────┘ │ </span><span> │ │ </span><span>┌──────────────┐ │ ┌──────────────┐ │ ┌──────────────┐ </span><span>│ Trusted │ ▼ │ │ ▼ │ │ </span><span>│ Interpreter │──Produces───▶│Trusted Binary├──Produces──▶│Trusted Binary│ </span><span>│ │ │ │ │ │ </span><span>└──────────────┘ └──────────────┘ └──────────────┘ </span></code></pre> <p>This process would continue forever to the right, where each trusted binary is the result of trusted sources. This then ties into topics like <a href="https://reproducible-builds.org/">reproducible builds</a> which assert that you can separately rebuild the sources and attain the same binary, showing the process can not have been tampered with.</p> <h2 id="but-does-it-really-work-like-that">But does it really work like that?</h2> <p>Outside of thought exercises, there is little evidence of these attacks being carried out in reality.</p> <p>Last year in 2020 we saw supply chain attacks such as the <a href="https://en.wikipedia.org/wiki/SolarWinds#2019%E2%80%932020_supply_chain_attacks">Solarwinds supply chain attacks</a> which was reported by <a href="https://www.fireeye.com/blog/products-and-services/2020/12/global-intrusion-campaign-leverages-software-supply-chain-compromise.html">Fireeye</a> as <em>&quot;Inserting malicious code into legitimate software updates for the Orion software that allow an attacker remote access into the victim's environment&quot;</em>. What's really interesting here was that no compiler was compromised in the process like our theoretical attack, but code was simply inserted and then subsequently was released.</p> <p>Tavis Ormandy in his blog <a href="https://blog.cmpxchg8b.com/2020/07/you-dont-need-reproducible-builds.html">You don't need reproducible builds</a> covers supply chain security, and examines why reproducible builds are not effective in the promises and claims they present. Importantly, Tavis discusses how trivial it is to insert &quot;bugdoors&quot;, or pieces of code that are malicious and will not be found, and can potentially be waved off as human error.</p> <p>Today, we don't even need bugdoors, with Microsoft Security Response Centre reporting that <a href="https://msrc-blog.microsoft.com/2019/07/16/a-proactive-approach-to-more-secure-code/">70% of vulnerabilities are memory safety issues</a>.</p> <p>No amount of reproducible builds or compiler bootstrapping chain can shield us from the reality that attackers today will target the softest area, and today that is security issues in our languages, and insecure configuration of supply chain infrastructure.</p> <p>We don't need backdoored compilers when we know that a security critical piece of software written in C is still exposed to the network.</p> <h2 id="but-lets-assume">But lets assume ...</h2> <p>Okay, so lets assume that backdoored compilers are a real risk for a moment. We need to establish a few things first to create our secure bootstrapping environment, and these requirements generally are extremely difficult to meet.</p> <p>We will need:</p> <ul> <li>Trusted Interpreter</li> <li>Trusted Sources</li> </ul> <p>This is the foundation, having these two trusted entities that we can use to begin the process. But what is &quot;trusted&quot;? How can we define that these items are truly trusted?</p> <p>One method could be to check the cryptographic signatures of the released source code, to validate that it is &quot;what was released&quot;, but this does not mean that the source code is free from backdoors/bugdoors which are the very thing we are attempting to shield ourselves from.</p> <p>What would be truly required here is a detailed and complete audit of all of the source code to these compilers, which would be a monumental task in and of itself. So today instead, we do not perform source code audits, and we <em>blindly trust</em> the providers of the source code as legitimate and having provided us tamper-free source code. We assert that blind trust through the validation of those cryptographic signatures. We blindly trust that they have vetted every commit and line of code, and they have not had their own source code supply chain compromised in some way to provide us this &quot;trusted source&quot;. This gives us a relationship with the producers of that source, that they are trustworthy and have performed vetting of code and their members with privileges, that they will &quot;do the right thing&quot;™.</p> <p>The second challenge is asserting trust in the interpreter. Where did this binary come from? How was it built? Were it's sources trusted? As one can imagine, this becomes a very deep rabbit hole when we want to chase it, but in reality the approach taken by todays linux distributions is that &quot;well we haven't been compromised to this point, so I guess this one is okay&quot; and we yolo build with it. We then create a root of trust in that one point in time, which then creates our bootstrapping chain of trust for future builds of subsequent trusted sources.</p> <h2 id="so-what-about-rust">So what about Rust?</h2> <p>Rust is interesting compared to something like C (clang/gcc), as the rust project not only provides signed sources, they also provide signed static binaries of their compiler. This is because unlike clang/gcc which have very long release lifecycles, rust is released every six weeks and to build version N of the compiler, requires version N or N - 1. This allows people who have missed a version to easily skip ahead without needing to build every intermediate version of the compiler.</p> <p>A frequent complaint is the difficulty to package rust because any time releases are missed, you must compile every intermediate version to adhere to the bootstrappable guidelines and principles to created a more &quot;trusted&quot; compiler.</p> <p>But just like any other humans, in order to save time, when we miss a version, we can use the rust language's provided signed binaries to reset the chain, allowing us to miss versions of rust, or to re-package older versions in some cases.</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span> ┌──────────────┐ ┌──────────────┐ </span><span> │ │ Trusted │ │ Trusted │ </span><span> Missed │ Sources │──────┐ │ Sources │──────┐ </span><span> Version! │ │ │ │ │ │ </span><span> │ └──────────────┘ │ └──────────────┘ │ </span><span> │ │ │ </span><span>┌──────────────┐ │ ┌──────────────┐ │ ┌──────────────┐ │ </span><span>│ │ │ │Trusted Binary│ ▼ │ │ ▼ </span><span>│Trusted Binary│ │ │ (from rust) ├──Produces──▶│Trusted Binary│──Produces───▶ ... </span><span>│ │ │ │ │ │ │ </span><span>└──────────────┘ │ └──────────────┘ └──────────────┘ </span></code></pre> <p>This process here is interesting because:</p> <ul> <li>Using the signed binary from rust-lang is actually <em>faster</em> since we can skip one compiler rebuild cycle due to being the same version as the sources</li> <li>It shows that the &quot;bootstrappable&quot; trust chain, does not actually matter since we frequently move our trust root to the released binary from rust, rather than building all intermediates</li> </ul> <p>Given this process, we must ask, what value do we have from trying to adhere to the bootstrappable principles with rust? We already root our trust in the rust project, meaning that because we blindly trust the sources <em>and</em> the static compiler, why would our resultant compiler be any more &quot;trustworthy&quot; just because we were the ones who compiled it?</p> <p>Beyond this the binaries that are issued by the rust project are used by thousands of people every day through tools like rustup. In reality, these have been proven time and time again that they are trusted to be able to run on mass deployments, and that the rust project has the ability and capability to respond to issues in their source code as well as the binaries they provide. They certainly have earned the trust of many people through this!</p> <p>So why do we keep assuming both that we are somehow more trustworthy than the rust project, but simultaneously they are fully trusted in the artefacts they provide to us?</p> <h2 id="contradictions">Contradictions</h2> <p>It is this contradiction that has made me rethink the process that we take to packaging rust in SUSE. I think we should bootstrap from upstream rust every release because the rust project are in a far better position to perform audits and respond to trust threats than part time package maintainers that are commonly part of Linux distributions.</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>│ ┌──────────────┐ │ ┌──────────────┐ </span><span>│ │ Trusted │ │ │ Trusted │ </span><span>│ │ Sources │──────┐ │ │ Sources │──────┐ </span><span>│ │ │ │ │ │ │ │ </span><span>│ └──────────────┘ │ │ └──────────────┘ │ </span><span>│ │ │ │ </span><span>│ ┌──────────────┐ │ ┌──────────────┐ │ ┌──────────────┐ │ ┌──────────────┐ </span><span>│ │Trusted Binary│ ▼ │ │ │ │Trusted Binary│ ▼ │ │ </span><span>│ │ (from rust) ├──Produces──▶│Trusted Binary│ │ │ (from rust) ├──Produces──▶│Trusted Binary│ </span><span>│ │ │ │ │ │ │ │ │ │ </span><span>│ └──────────────┘ └──────────────┘ │ └──────────────┘ └──────────────┘ </span></code></pre> <p>We already fully trust the sources they release, and we already fully trust their binary compiler releases. We can simplify our build process (and speed it up!) by acknowledging this trust relationship exists, rather than trying to continue to convince ourselves that we are somehow &quot;more trusted&quot; than the rust project.</p> <p>Also we must consider the reality of threats in the wild. Does all of this work and discussions of who is more trusted really pay off and defend us in reality? Or are we focused on these topics because they are something that we can control and have opinions over, rather than acknowledging the true complexity and dirtiness of security threats as they truly exist today?</p> Open Source Enshrines the Wrong Privilege Tue, 23 Mar 2021 00:00:00 +0000 Unknown https://fy.blackhats.net.au/blog/2021-03-23-open-source-enshrines-the-wrong-privilege/ https://fy.blackhats.net.au/blog/2021-03-23-open-source-enshrines-the-wrong-privilege/ <h1 id="open-source-enshrines-the-wrong-privilege">Open Source Enshrines the Wrong Privilege</h1> <p>Within Open Source/Free Software, we repeatedly see a set of behaviours - hostile or toxic project owners, abusive relationships, aggression towards users, and complete disregard to users of the software. Some projects have risen above this and advanced the social behaviours in their communities, but these are still the minority of projects.</p> <p>Many advocates for FLOSS have been trying to enhance adoption of these technologies in communities, but with the exception of limited non-technical audiences, this really hasn't gained much ground.</p> <p>It is my opinion that these community behaviours, and the low adoption of FLOSS technologies comes back to what our Open Source licenses enshrine - the very thing they embody and create.</p> <h2 id="the-origins-of-free-software">The Origins of Free Software</h2> <p>The story of Free Software starts with an individual (later revealed as abusive), who was frustrated at not being able to access software on a printer so that he could alter it's behaviour. This has been extended to the idea that Free Software &quot;grants people control over their own lives and software&quot;.</p> <p>This however, is not correct.</p> <p>What Free Software licenses protect is that individuals with time, resources, specialised technical knowledge and social standing have the possibility to alter that software's behaviour.</p> <p>When we consider that the majority of the world are not developers or software engineers, what is it that our Free Software is doing to protect and support these individuals? Should we truly expect individuals who are linguists, authors, scientists, retail staff, or social workers to be able to &quot;alter the software to fix their own problems&quot;?</p> <p>Even as technical experts, we are frustrated when someone closes an issue with &quot;PR's welcome&quot;. Imagine how these other people feel when they can't even express or report the problem in the first place or get told they aren't good enough, or that &quot;they can fix it themselves if they want&quot;.</p> <p>This attitude also discounts the subject matter knowledge required to alter or contribute to any piece of software however. I may be a Senior Software Engineer, but I lack the knowledge and time to contribute to Gnome for example. Even with these &quot;freedoms&quot; I lack the ability to &quot;control&quot; the software on my own system.</p> <h2 id="open-source-is-selfish">Open Source is Selfish</h2> <p>These licenses that we have in FLOSS all enshrine selfish and privileged behaviours.</p> <p><strong>I</strong> have the rights to freely access this code so <strong>I</strong> can read it or alter it.</p> <p><strong>I</strong> can change this project to fix issues <strong>I</strong> have.</p> <p><strong>I</strong> have freedoms.</p> <p>None of these statements from FLOSS describe other people - the people who consume our software (in some cases, without choice). People who are not subject matter experts and can't contribute to &quot;solve their own problems&quot;. People who may not have the experience and language to describe the problems they face.</p> <p>This lack of empathy, the lack of concern for others in FLOSS leads us to where we are now. Those who have the subject matter knowledge lead projects, and do what <em>they</em> want because <em>they</em> can fix it. They tell others &quot;PR's welcome&quot; knowing full-well that the other person may never be able to contribute, that the barriers to contribution are so high (both in programming experience and domain knowledge). They design the software to work the way they want, because they understand it and it &quot;works for me&quot;.</p> <p>This is reflected in our software. Software that not does not care for the needs, experiences or rights of others. Software that pretends to be accessible, all while creating gated communities of control. Software that is out of reach of people, the same people that we &quot;claim&quot; to be working for and supporting.</p> <p>It leads to our communities that are selfish, and do not empathise with people. Communities that have placed negative behaviours on pedestals and turned these people into &quot;leaders&quot;. Software that does not account for the experiences of our users, believing that the &quot;community knows best&quot;.</p> <p>One does not need to look far for FLOSS projects that speak one set of words, but their actions do not align.</p> <h2 id="what-can-we-do">What Can We Do?</h2> <p>In our projects we need to go beyond preserving the freedoms of ourselves, and begin to discuss the freedoms and interactions that others should have with our systems and projects. Here are some starting ideas that I have:</p> <ul> <li>Have a code of conduct for all contributors (remember, opening an issue is a contribution).</li> <li>Document your target users, and what kind of experience they should have. Expand this over time.</li> <li>Promote empathy for those who aren't direct contributors - indirect users without choice exist.</li> <li>Remove dependencies on as many problematic software projects as possible.</li> <li>Push for improvements to open licenses that enshrine the freedoms of others - not just developers.</li> </ul> <p>As individual communities we can advance the state of software and how we act socially so that future projects and users are in a better place. No software exists in a vacuum, all software exists to support people. We need to always keep in mind the effects our software has on others.</p> Time Machine on Samba with ZFS Mon, 22 Mar 2021 00:00:00 +0000 Unknown https://fy.blackhats.net.au/blog/2021-03-22-time-machine-on-samba-with-zfs/ https://fy.blackhats.net.au/blog/2021-03-22-time-machine-on-samba-with-zfs/ <h1 id="time-machine-on-samba-with-zfs">Time Machine on Samba with ZFS</h1> <p>Time Machine is Apple's in-built backup system for MacOS. It's probably the best consumer backup option, which really achieves &quot;set and forget&quot; backups.</p> <p>It can backup to an external hard disk on a dock, an Apple Time Machine (wireless access point), or a custom location based on SMB shares.</p> <p>Since I have a fileserver at home, I use this as my Time Machine backup target. To make this work really smoothly there are a few setup steps.</p> <h2 id="macos-time-machine-performance">MacOS Time Machine Performance</h2> <p>By default timemachine operates as a low priority process. You can set a sysctl to improve the performance of this (especially helpful for a first backup!)</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>sysctl -w debug.lowpri_throttle_enabled=0 </span></code></pre> <p>You will need a launchd script to make this setting survive a reboot.</p> <h2 id="zfs">ZFS</h2> <p>I'm using ZFS on my server, which is probably the best filesystem available. To make Time Machine work well on ZFS there are a number of tuning options that can help. As these backups write and read many small files, you should have a large amount of RAM for ARC (best) or a ZIL on nvme. RAID 10 will likely work better than RAIDZ here as you need better seek latency than write throughput due to the need to access many small files. Generally time machine is very &quot;IO demanding&quot;.</p> <p>For the ZFS properties on the filesystem I created it with the following options to [zfs create]{.title-ref}. Each once is set with [-o attribute=value]{.title-ref}</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>atime: off </span><span>dnodesize: auto </span><span>xattr: sa </span><span>logbias: throughput </span><span>recordsize: 1M </span><span>compression: zstd-10 | zle </span><span>refquota: 3T </span><span># optional - greatly improves write performance </span><span>sync: disabled </span><span># security </span><span>setuid: off </span><span>exec: off </span><span>devices: off </span></code></pre> <p>The important ones here are the compression setting. If you choose zle, you gain much faster write performance, but you dont get much in the way of compression. zstd-10 gives me about 1.3x compression, but at the loss of performance. Generally the decision is based on your pool and storage capacity.</p> <p>Also note the use of refquota instead of quota. This applies the quota to this filesystem only excluding snapshots - if you use quota, the space taken by snapshots it also applied to this filesystem, which may cause you to run out of space.</p> <p>You may optionally choose to disable sync. This is because Time Machine issues a sync after every single file write to the server, which can cause low performance with many small files. To mitigate the data loss risk here, I snapshot the backups filesystem hourly.</p> <p>If you want to encrypt at the ZFS level instead of through time machine you need to enable this as you create the filesystem.</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span># create a key file to unlock the zfs filesystem </span><span>openssl rand -hex -out /root/key 32 </span><span> </span><span># Add the following settings during zfs create: </span><span>-o encryption=aes-128-gcm -o keyformat=hex -o keylocation=file:///root/key </span></code></pre> <p>If you add any subvolumes, you need to repeat the same encryption steps during the create of these subvolumes.</p> <p>For example a create may look like:</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>zfs create \ </span><span> -o encryption=aes-128-gcm -o keyformat=hex -o keylocation=file:///root/key \ </span><span> -o atime=off -o dnodesize=auto -o xattr=sa -o logbias=throughput \ </span><span> -o recordsize=1M -o compression=zle -o refquota=3T -o sync=disabled \ </span><span> -o setuid=off -o exec=off -o devices=off tank/backups </span></code></pre> <h2 id="smb-conf">smb.conf</h2> <p>In smb.conf you define the share that exposes the timemachine backup location. You need to set additional metadata on this so that macos will recognise it correctly.</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>[global] </span><span>min protocol = SMB2 </span><span>ea support = yes </span><span> </span><span># This needs to be global else time machine ops can fail. </span><span>vfs objects = fruit streams_xattr </span><span>fruit:aapl = yes </span><span>fruit:metadata = stream </span><span>fruit:model = MacSamba </span><span>fruit:posix_rename = yes </span><span>fruit:veto_appledouble = no </span><span>fruit:nfs_aces = no </span><span>fruit:wipe_intentionally_left_blank_rfork = yes </span><span>fruit:delete_empty_adfiles = yes </span><span>spotlight = no </span><span> </span><span>[timemachine_a] </span><span>comment = Time Machine </span><span>fruit:time machine = yes </span><span>fruit:time machine max size = 1050G </span><span>path = /var/data/backup/timemachine_a </span><span>browseable = yes </span><span>write list = timemachine </span><span>create mask = 0600 </span><span>directory mask = 0700 </span><span># NOTE: Changing these will require a new initial backup cycle if you already have an existing </span><span># timemachine share. </span><span>case sensitive = true </span><span>default case = lower </span><span>preserve case = no </span><span>short preserve case = no </span></code></pre> <p>The fruit settings are required to help Time Machine understand that this share is usable for it. I have also added a custom timemachine user to smbpasswd, and created a matching posix account who should own these files.</p> <h2 id="macos">MacOS</h2> <p>You can now add this to MacOS via system preferences. If your ZFS volume is NOT encyrpted, you should add the timemachine volume via system preferences, as it is the only way to enable encryption of the time machine backup. For system preferences to &quot;see&quot; the samba share you may need to mount it manually via finder as the time machine user.</p> <p>If you are using ZFS encryption, you can add the time machine backup from the command line instead.</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>tmutil setdestination smb://timemachine:password@hostname/timemachine_a </span></code></pre> <p>If you intend to have multiple time machine targets, MacOS is capable of mirroring between multilple stripes alternately. You can append the second stripe with (note the -a). You could do this with other shares (offsite for example) or with a HDD on your desk.</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>tmutil setdestination -a smb://timemachine:password@hostname/timemachine_b </span></code></pre> Against Packaging Rust Crates Tue, 16 Feb 2021 00:00:00 +0000 Unknown https://fy.blackhats.net.au/blog/2021-02-16-against-packaging-rust-crates/ https://fy.blackhats.net.au/blog/2021-02-16-against-packaging-rust-crates/ <h1 id="against-packaging-rust-crates">Against Packaging Rust Crates</h1> <p>Recently the discussion has once again come up around the notion of packaging Rust crates as libraries in distributions. For example, taking a library like [serde]{.title-ref} and packaging it to an RPM. While I use RPM as the examples here it applies equally to other formats.</p> <p>Proponents of crate packaging want all Rust applications to use the &quot;distributions&quot; versions of a crate. This is to prevent &quot;vendoring&quot; or &quot;bundling&quot;. This is where an application (such as 389 Directory Server) ships all of it's sources, as well as the sources of it's Rust dependencies in a single archive. These sources may differ in version from the bundled sources of other applications.</p> <h2 id="packaging-crates-is-not-reinventing-cargo">&quot;Packaging crates is not reinventing Cargo&quot;</h2> <p>This is a common claim by advocates of crate packaging. However it is easily disproved:</p> <p><em>If packaging is not reinventing cargo, I am free to use all of Cargo's features without conflicts to distribution packaging.</em></p> <p>The reality is that packaging crates <em>is</em> reinventing Cargo - but without all it's features. Common limitations are that Cargo's exact version/less than requirements can not be used safely, or Cargo's ability to apply patches or uses sources from specific git revisions can not be used at all.</p> <p>As a result, this hinders upstreams from using all the rich features within Cargo to comply with distribution packaging limitations, or it will cause the package to hit exceptions in policy and necesitate vendoring anyway.</p> <h2 id="you-can-vendor-only-in-these-exceptional-cases">&quot;You can vendor only in these exceptional cases ...&quot;</h2> <p>As noted, since packaging is reinventing Cargo, if you use features of Cargo that are unsupported then you may be allowed to vendor depending on the distributions policy. However, this raises some interesting issues itself.</p> <p>Assume I have been using distribution crates for a period of time - then the upstream adds an exact version or git revision requirement to a project or a dependency in my project. I now need to change my spec file and tooling to use vendoring and all of the benefits of distribution crates no longer exists (because you can not have any dependency in your tree that has an exact version rule).</p> <p>If the upstream 'un-does' that change, then I need to roll back to distribution crates since the project would no longer be covered by the exemption.</p> <p>This will create review delays and large amounts of administrative overhead. It means pointless effort to swap between vendored and distribution crates based on small upstream changes. This may cause packagers to avoid certain versions or updates so that they do not need to swap between distribution methods.</p> <p>It's very likely that these &quot;exceptional&quot; cases will be very common, meaning that vendoring will be occuring. This necesitates supporting vendored applications in distribution packages.</p> <h2 id="you-don-t-need-to-package-the-universe">&quot;You don't need to package the universe&quot;</h2> <p>Many proponents say that they have &quot;already packaged most things&quot;. For example in 389 Directory Server of our 60 dependencies, only 2 were missing in Fedora (2021-02). However this overlooks the fact that I do not want to package those 2 other crates just to move forward. I want to support 389 Directory Server the <em>application</em> not all of it's dependencies in a distribution.</p> <p>This is also before we come to larger rust projects, such as Kanidm that has nearly 400 dependencies. The likelihood that many of them are missing is high.</p> <p>So you will need to package the universe. Maybe not all of it. But still a lot of it. It's already hard enough to contribute packages to a distribution. It becomes even harder when I need to submit 3, 10, or 100 more packages. It could be months before enough approvals were in place. It's a staggering amount of administration and work, which will discourage many contributors.</p> <p>People have already contacted me to say that if they had to package crates to distribution packages to contribute, they would give up and walk away. We've already lost future contributors.</p> <p>Further to this Ruby, Python and many other languages today all recommend language native tools such as rvm or virtualenv to avoid using distribution packaged libraries.</p> <p>Packages in distributions should exist as a vehicle to ship bundled applications that are created from their language native tools.</p> <h2 id="we-will-update-your-dependencies-for-you">&quot;We will update your dependencies for you&quot;</h2> <p>A supposed benefit is that versions of crates in distributions will be updated in the background according to semver rules.</p> <p>If we had an exact version requirement (that was satisfiable), a silent background update will cause this to no longer work - and will break the application from building. This would necesitate one of:</p> <ul> <li>A change to the Cargo.toml to remove the equality requirement - a requirement that may exist for good reason.</li> <li>It will force the application to temporarily swap to vendoring instead.</li> <li>The application will remain broken and unable to be updated until upstream resolves the need for the equality requirement.</li> </ul> <p>Background updates also ignore the state of your Cargo.lock file by removing it. A Cargo.lock file is recommended to be checked in with binary applications in Rust, as evidence that shows &quot;here is an exact set of dependencies that upstream has tested and verified as building and working&quot;.</p> <p>To remove and ignore this file, means to remove the guarantees of quality from an upstream.</p> <p>It is unlikely that packagers will run the entire test suite of an application to regain this confidence. They will &quot;apply the patch and pray&quot; method - as they already do with other languages.</p> <p>We can already see how background updates can have significant negative consequences on application stability. FreeIPA has hundreds of dependencies, and it's common that if any of them changes in small ways, it can cause FreeIPA to fall over. This is not the fault of FreeIPA - it's the fault of relying on so many small moving parts that can change underneath your feet without warning. FreeIPA would strongly benefit from vendoring to improve it's stability and quality.</p> <p>Inversely, it can cause hesitation to updating libraries - since there is now a risk of breaking other applications that depend on them. We do not want people to be afraid of updates.</p> <h2 id="we-can-respond-to-security-issues">&quot;We can respond to security issues&quot;</h2> <p>On the surface this is a strong argument, but in reality it does not hold up. The security issues that face Rust are significantly different to that which affect C. In C it may be viable to patch and update a dynamic library to fix an issue. It saves time because you only need to update and change one library to fix everything.</p> <p>Security issues are much rarer in Rust. When they occur, you will have to update and re-build all applications depending on the affected library.</p> <p>Since this rebuilding work has to occur, where the security fix is applied is irrelevant. This frees us to apply the fixes in a different way to how we approach C.</p> <p>It is better to apply the fixes in a consistent and universal manner. There <em>will</em> be applications that are vendored due to vendoring exceptions, there is now duplicated work and different processes to respond to both distribution crates, and vendored applications.</p> <p>Instead all applications could be vendored, and tooling exists that would examine the Cargo.toml to check for insecure versions (RustSec/cargo-audit does this for example). The Cargo.toml's can be patched, and applications tested and re-vendored. Even better is these changes could easily then be forwarded to upstreams, allowing every distribution and platform to benefit from the work.</p> <p>In the cases that the upstream can not fix the issue, then Cargo's native patching tooling can be used to supply fixes directly into vendored sources for rare situations requiring it.</p> <h2 id="patching-20-vulnerable-crates-doesn-t-scale-we-need-to-patch-in-one-place">&quot;Patching 20 vulnerable crates doesn't scale, we need to patch in one place!&quot;</h2> <p>A common response to the previous section is that the above process won't scale as we need to find and patch 20 locations compared to just one. It will take &quot;more human effort&quot;.</p> <p>Today, when a security fix comes out, every distribution's security teams will have to be made aware of this. That means - OpenSUSE, Fedora, Debian, Ubuntu, Gentoo, Arch, and many more groups all have to become aware and respond. Then each of these projects security teams will work with their maintainers to build and update these libraries. In the case of SUSE and Red Hat this means that multiple developers may be involved, quality engineering will be engaged to test these changes. Consumers of that library will re-test their applications in some cases to ensure there are no faults of the components they rely upon. This is all before we approach the fact that each of these distributions have many supported and released versions they likely need to maintain so this process may be repeated for patching and testing multiple versions in parallel.</p> <p>In this process there are a few things to note:</p> <ul> <li>There is a huge amount of human effort today to keep on top of security issues in our distributions.</li> <li>Distributions tend to be isolated and can't share the work to resolve these - the changes to the rpm specs in SUSE won't help Debian for example.</li> <li>Human error occurs in all of these layers causing security issues to go un-fixed or breaking a released application.</li> </ul> <p>To suggest that rust and vendoring somehow makes this harder or more time consuming is discounting the huge amount of time, skill, and effort already put in by people to keep our C based distributions functioning today.</p> <p>Vendored Rust won't make this process easier or harder - it just changes the nature of the effort we have to apply as maintainers and distributions. It shifts our focus from &quot;how do we ensure this library is secure&quot; to &quot;how do we ensure this <em>application</em> made from many libraries is secure&quot;. It allows further collaboration with upstreams to be involved in the security update process, which ends up benefiting <em>all</em> distributions.</p> <h2 id="it-doesn-t-duplicate-effort">&quot;It doesn't duplicate effort&quot;</h2> <p>It does. By the very nature of both distribution libraries and vendored applications needing to exist in a distribution, there will become duplicated but seperate processes and policies to manage these, inspect, and update these. This will create a need for tooling and supporting both methods, which consumes time for many people.</p> <p>People have already done the work to package and release libraries to crates.io. Tools already exist to provide our dependencies and include them in our applications. Why do we need to duplicate these features and behaviours in distribution packages when Cargo already does this correctly, and in a way that is universal and supported.</p> <h2 id="don-t-support-distribution-crates">Don't support distribution crates</h2> <p>I can't be any clearer than that. They consume excessive amounts of contributor time, for little to no benefit, it detracts from simpler language-native solutions for managing dependencies, distracts from better language integration tooling being developed, it can introduce application instability and bugs, and it creates high barriers to entry for new contributors to distributions.</p> <p>It doesn't have to be like this.</p> <p>We need to stop thinking that Rust is like C. We have to accept that language native tools are the interface people will want to use to manage their libraries and distribute whole applications. We must use our time more effectively as distributions.</p> <p>If we focus on supporting vendored Rust applications, and developing our infrastructure and tooling to support this, we <em>will</em> attract new contributors by lowering barriers to entry, but we will also have a stronger ability to contribute back to upstreams, and we will simplify our building and packaging processes.</p> <p>Today, tools like docker, podman, flatpak, snapd and others have proven how bundling/vendoring, and a focus an applications can advance the state of our ecosystems. We need to adopt the same ideas into distributions. Our package managers should become a method to ship applications - not libraries.</p> <p>We need to focus our energy to supporting <em>applications</em> as self contained units - not supporting the libraries that make them up.</p> <h2 id="edits">Edits</h2> <ul> <li>Released: 2021-02-16</li> <li>EDIT: 2021-02-22 - improve clarity on some points, thanks to ftweedal.</li> <li>EDIT: 2021-02-23 - due to a lot of comments regarding security updates, added an extra section to address how this scales.</li> </ul> Getting Started Packaging A Rust CLI Tool in SUSE OBS Mon, 15 Feb 2021 00:00:00 +0000 Unknown https://fy.blackhats.net.au/blog/2021-02-15-getting-started-packaging-a-rust-cli-tool-in-suse-obs/ https://fy.blackhats.net.au/blog/2021-02-15-getting-started-packaging-a-rust-cli-tool-in-suse-obs/ <h1 id="getting-started-packaging-a-rust-cli-tool-in-suse-obs">Getting Started Packaging A Rust CLI Tool in SUSE OBS</h1> <p>Distribution packaging always seems like something that is really difficult or hard to do, but the SUSE <a href="https://build.opensuse.org">Open Build Service</a> makes it really easy to not only build packages, but to then contribute them to Tumbleweed. Not only that, OBS can also build for Fedora, CentOS and more.</p> <h2 id="getting-started">Getting Started</h2> <p>You'll need to sign up to service - there is a sign up link on the front page of <a href="https://build.opensuse.org">OBS</a></p> <p>To do this you'll need a SUSE environment. Docker is an easy way to create this without having to commit to a full virtual machine / install.</p> <pre data-lang="bash" style="background-color:#2b303b;color:#c0c5ce;" class="language-bash "><code class="language-bash" data-lang="bash"><span> </span><span style="color:#bf616a;">docker</span><span> run \ </span><span style="color:#bf616a;"> --security-opt</span><span>=seccomp:unconfined</span><span style="color:#bf616a;"> --cap-add</span><span>=SYS_PTRACE</span><span style="color:#bf616a;"> --cap-add</span><span>=SYS_CHROOT</span><span style="color:#bf616a;"> --cap-add</span><span>=SYS_ADMIN \ </span><span style="color:#bf616a;"> -i -t</span><span> opensuse/tumbleweed:latest /bin/sh </span><span style="color:#bf616a;">bash </span><span> </span><span style="color:#bf616a;">-</span><span> NOTE: We need these extra privileges so that the osc build command </span><span> </span><span style="color:#bf616a;">can</span><span> work due to how it uses chroots/mounts. </span><span> </span><span style="color:#bf616a;">Inside</span><span> of this we</span><span style="color:#96b5b4;">\&#39;</span><span>ll need some packages to help make the process </span><span style="color:#bf616a;">easier. </span><span> </span><span>```</span><span style="color:#bf616a;">bash </span><span> </span><span style="color:#bf616a;">zypper</span><span> install obs-service-cargo_vendor osc obs-service-tar obs-service-obs_scm \ </span><span> obs-service-recompress obs-service-set_version obs-service-format_spec_file \ </span><span> obs-service-cargo_audit cargo sudo </span></code></pre> <p>You should also install your editor of choice in this command (docker images tend not to come with any editors!)</p> <p>You'll need to configure osc, which is the CLI interface to OBS. This is done in the file [~/.config/osc/oscrc]{.title-ref}. A minimal starting configuration is:</p> <pre data-lang="ini" style="background-color:#2b303b;color:#c0c5ce;" class="language-ini "><code class="language-ini" data-lang="ini"><span style="color:#b48ead;"> [general] </span><span> </span><span style="color:#65737e;"># URL to access API server, e.g. https://api.opensuse.org </span><span> </span><span style="color:#65737e;"># you also need a section [https://api.opensuse.org] with the credentials </span><span> </span><span style="color:#bf616a;">apiurl </span><span>= </span><span style="color:#d08770;">https://api.opensuse.org </span><span style="color:#b48ead;"> [https://api.opensuse.org] </span><span> </span><span style="color:#bf616a;">user </span><span>= &lt;username&gt; </span><span> </span><span style="color:#bf616a;">pass </span><span>= &lt;password&gt; </span></code></pre> <p>You can check this works by using the &quot;whois&quot; command.</p> <pre data-lang="bash" style="background-color:#2b303b;color:#c0c5ce;" class="language-bash "><code class="language-bash" data-lang="bash"><span> </span><span style="color:#65737e;"># osc whois </span><span> </span><span style="color:#bf616a;">firstyear: </span><span>&quot;</span><span style="color:#a3be8c;">William Brown</span><span>&quot; &lt;email here&gt; </span></code></pre> <p>Optionally, you may install <a href="https://github.com/Firstyear/cargo-lock2rpmprovides">cargo lock2rpmprovides</a> to assist with creation of the license string for your package:</p> <pre data-lang="bash" style="background-color:#2b303b;color:#c0c5ce;" class="language-bash "><code class="language-bash" data-lang="bash"><span> </span><span style="color:#bf616a;">cargo</span><span> install cargo-lock2rpmprovides </span></code></pre> <h2 id="packaging-a-rust-project">Packaging A Rust Project</h2> <p>In this example we'll use a toy Rust application I created called <a href="https://github.com/Firstyear/hellorust">hellorust</a>. Of course, feel free to choose your own project or Rust project you want to package!</p> <ul> <li>HINT: It's best to choose binaries, not libraries to package. This is because Rust can self-manage it's dependencies, so we don't need to package every library. Neat!</li> </ul> <p>First we'll create a package in our OBS home project.</p> <pre data-lang="bash" style="background-color:#2b303b;color:#c0c5ce;" class="language-bash "><code class="language-bash" data-lang="bash"><span> </span><span style="color:#bf616a;">osc</span><span> co home:&lt;username&gt; </span><span> </span><span style="color:#96b5b4;">cd</span><span> home:&lt;username&gt; </span><span> </span><span style="color:#bf616a;">osc</span><span> mkpac hellorust </span><span> </span><span style="color:#96b5b4;">cd</span><span> hellorust </span></code></pre> <p>OBS comes with a lot of useful utilities to help create and manage sources for our project. First we'll create a skeleton RPM spec file. This should be in a file named [hellorust.spec]{.title-ref}</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span> %global rustflags -Clink-arg=-Wl,-z,relro,-z,now -C debuginfo=2 </span><span> </span><span> Name: hellorust </span><span> # This will be set by osc services, that will run after this. </span><span> Version: 0.0.0 </span><span> Release: 0 </span><span> Summary: A hello world with a number of the day printer. </span><span> # If you know the license, put it&#39;s SPDX string here. </span><span> # Alternately, you can use cargo lock2rpmprovides to help generate this. </span><span> License: Unknown </span><span> # Select a group from this link: </span><span> # https://en.opensuse.org/openSUSE:Package_group_guidelines </span><span> Group: Amusements/Games/Other </span><span> Url: https://github.com/Firstyear/hellorust </span><span> Source0: %{name}-%{version}.tar.xz </span><span> Source1: vendor.tar.xz </span><span> Source2: cargo_config </span><span> </span><span> BuildRequires: rust-packaging </span><span> ExcludeArch: s390 s390x ppc ppc64 ppc64le %ix86 </span><span> </span><span> %description </span><span> A hello world with a number of the day printer. </span><span> </span><span> %prep </span><span> %setup -q </span><span> %setup -qa1 </span><span> mkdir .cargo </span><span> cp %{SOURCE2} .cargo/config </span><span> # Remove exec bits to prevent an issue in fedora shebang checking </span><span> find vendor -type f -name \*.rs -exec chmod -x &#39;{}&#39; \; </span><span> </span><span> %build </span><span> export RUSTFLAGS=&quot;%{rustflags}&quot; </span><span> cargo build --offline --release </span><span> </span><span> %install </span><span> install -D -d -m 0755 %{buildroot}%{_bindir} </span><span> </span><span> install -m 0755 %{_builddir}/%{name}-%{version}/target/release/hellorust %{buildroot}%{_bindir}/hellorust </span><span> </span><span> %files </span><span> %{_bindir}/hellorust </span><span> </span><span> %changelog </span></code></pre> <p>There are a few commented areas you'll need to fill in and check. But next we will create a service file that allows OBS to help get our sources and bundle them for us. This should go in a file called <code>_service</code></p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span> &lt;services&gt; </span><span> &lt;service mode=&quot;disabled&quot; name=&quot;obs_scm&quot;&gt; </span><span> &lt;!-- ✨ URL of the git repo ✨ --&gt; </span><span> &lt;param name=&quot;url&quot;&gt;https://github.com/Firstyear/hellorust.git&lt;/param&gt; </span><span> &lt;param name=&quot;versionformat&quot;&gt;@PARENT_TAG@~git@TAG_OFFSET@.%h&lt;/param&gt; </span><span> &lt;param name=&quot;scm&quot;&gt;git&lt;/param&gt; </span><span> &lt;!-- ✨ The version tag or branch name from git ✨ --&gt; </span><span> &lt;param name=&quot;revision&quot;&gt;v0.1.1&lt;/param&gt; </span><span> &lt;param name=&quot;match-tag&quot;&gt;*&lt;/param&gt; </span><span> &lt;param name=&quot;versionrewrite-pattern&quot;&gt;v(\d+\.\d+\.\d+)&lt;/param&gt; </span><span> &lt;param name=&quot;versionrewrite-replacement&quot;&gt;\1&lt;/param&gt; </span><span> &lt;param name=&quot;changesgenerate&quot;&gt;enable&lt;/param&gt; </span><span> &lt;!-- ✨ Your email here ✨ --&gt; </span><span> &lt;param name=&quot;changesauthor&quot;&gt; YOUR EMAIL HERE &lt;/param&gt; </span><span> &lt;/service&gt; </span><span> &lt;service mode=&quot;disabled&quot; name=&quot;tar&quot; /&gt; </span><span> &lt;service mode=&quot;disabled&quot; name=&quot;recompress&quot;&gt; </span><span> &lt;param name=&quot;file&quot;&gt;*.tar&lt;/param&gt; </span><span> &lt;param name=&quot;compression&quot;&gt;xz&lt;/param&gt; </span><span> &lt;/service&gt; </span><span> &lt;service mode=&quot;disabled&quot; name=&quot;set_version&quot;/&gt; </span><span> &lt;service name=&quot;cargo_audit&quot; mode=&quot;disabled&quot;&gt; </span><span> &lt;!-- ✨ The name of the project here ✨ --&gt; </span><span> &lt;param name=&quot;srcdir&quot;&gt;hellorust&lt;/param&gt; </span><span> &lt;/service&gt; </span><span> &lt;service name=&quot;cargo_vendor&quot; mode=&quot;disabled&quot;&gt; </span><span> &lt;!-- ✨ The name of the project here ✨ --&gt; </span><span> &lt;param name=&quot;srcdir&quot;&gt;hellorust&lt;/param&gt; </span><span> &lt;param name=&quot;compression&quot;&gt;xz&lt;/param&gt; </span><span> &lt;/service&gt; </span><span> </span><span> &lt;/services&gt; </span></code></pre> <p>Now this service file does a lot of the heavy lifting for us:</p> <ul> <li>It will fetch the sources from git, based on the version we set.</li> <li>It will turn them into a tar.xz for us.</li> <li>It will update the changelog for the rpm, and set the correct version in the spec file.</li> <li>It scans our project for any known vulnerabilities</li> <li>It will download our rust dependencies, and then bundle them to vendor.tar.xz.</li> </ul> <p>So our current work dir should look like:</p> <pre data-lang="bash" style="background-color:#2b303b;color:#c0c5ce;" class="language-bash "><code class="language-bash" data-lang="bash"><span> </span><span style="color:#65737e;"># ls -1 . </span><span> </span><span style="color:#bf616a;">.osc </span><span> </span><span style="color:#bf616a;">_service </span><span> </span><span style="color:#bf616a;">hellorust.spec </span></code></pre> <p>Now we can run [osc service ra]{.title-ref}. This will run the services in our [_service]{.title-ref} file as we mentioned. Once it's complete we'll have quite a few more files in our directory:</p> <pre data-lang="bash" style="background-color:#2b303b;color:#c0c5ce;" class="language-bash "><code class="language-bash" data-lang="bash"><span> </span><span style="color:#65737e;"># ls -1 . </span><span> </span><span style="color:#bf616a;">_service </span><span> </span><span style="color:#bf616a;">_servicedata </span><span> </span><span style="color:#bf616a;">cargo_config </span><span> </span><span style="color:#bf616a;">hellorust </span><span> </span><span style="color:#bf616a;">hellorust-0.1.1~git0.db340ad.obscpio </span><span> </span><span style="color:#bf616a;">hellorust-0.1.1~git0.db340ad.tar.xz </span><span> </span><span style="color:#bf616a;">hellorust.obsinfo </span><span> </span><span style="color:#bf616a;">hellorust.spec </span><span> </span><span style="color:#bf616a;">vendor.tar.xz </span></code></pre> <p>Inside the [hellorust]{.title-ref} folder ([home:username/hellorust/hellorust]{.title-ref}), is a checkout of our source. If you cd to that directory, you can run [cargo lock2rpmprovides]{.title-ref} which will display your license string you need:</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span> License: ( Apache-2.0 OR MIT ) AND ( Apache-2.0 WITH LLVM-exception OR Apache-2.0 OR MIT ) AND </span></code></pre> <p>Just add the license from the project, and then we can update our [hellorust.spec]{.title-ref} with the correct license.</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span> License: ( Apache-2.0 OR MIT ) AND ( Apache-2.0 WITH LLVM-exception OR Apache-2.0 OR MIT ) AND MPL-2.0 </span></code></pre> <ul> <li>HINT: You don't need to use the emitted &quot;provides&quot; lines here. They are just for fedora rpms to adhere to some of their policy requirements.</li> </ul> <p>Now we can build our package on our local system to test it. This may take a while to get all its build dependencies and other parts, so be patient :)</p> <pre data-lang="bash" style="background-color:#2b303b;color:#c0c5ce;" class="language-bash "><code class="language-bash" data-lang="bash"><span> </span><span style="color:#bf616a;">osc</span><span> build </span></code></pre> <p>If that completes successfully, you can now test these rpms:</p> <pre data-lang="bash" style="background-color:#2b303b;color:#c0c5ce;" class="language-bash "><code class="language-bash" data-lang="bash"><span> </span><span style="color:#65737e;"># zypper in /var/tmp/build-root/openSUSE_Tumbleweed-x86_64/home/abuild/rpmbuild/RPMS/x86_64/hellorust-0.1.1~git0.db340ad-0.x86_64.rpm </span><span> (</span><span style="color:#bf616a;">1/1</span><span>) Installing: hellorust-0.1.1</span><span style="color:#bf616a;">~</span><span>git0.db340ad-0.x86_64 ... </span><span style="color:#b48ead;">[</span><span>done</span><span style="color:#b48ead;">] </span><span> </span><span style="color:#65737e;"># rpm -ql hellorust </span><span> </span><span style="color:#bf616a;">/usr/bin/hellorust </span><span> </span><span style="color:#65737e;"># hellorust </span><span> </span><span style="color:#bf616a;">Hello,</span><span> Rust! The number of the day is: 68 </span></code></pre> <p>Next you can commit to your project. Add the files that we created:</p> <pre data-lang="bash" style="background-color:#2b303b;color:#c0c5ce;" class="language-bash "><code class="language-bash" data-lang="bash"><span> </span><span style="color:#65737e;"># osc add _service cargo_config hellorust-0.1.1~git0.db340ad.tar.xz hellorust.spec vendor.tar.xz </span><span> </span><span style="color:#65737e;"># osc status </span><span> </span><span style="color:#bf616a;">A</span><span> _service </span><span> </span><span style="color:#bf616a;">?</span><span> _servicedata </span><span> </span><span style="color:#bf616a;">A</span><span> cargo_config </span><span> </span><span style="color:#bf616a;">?</span><span> hellorust-0.1.1</span><span style="color:#bf616a;">~</span><span>git0.db340ad.obscpio </span><span> </span><span style="color:#bf616a;">A</span><span> hellorust-0.1.1</span><span style="color:#bf616a;">~</span><span>git0.db340ad.tar.xz </span><span> </span><span style="color:#bf616a;">?</span><span> hellorust.obsinfo </span><span> </span><span style="color:#bf616a;">A</span><span> hellorust.spec </span><span> </span><span style="color:#bf616a;">A</span><span> vendor.tar.xz </span></code></pre> <ul> <li>HINT: You DO NOT need to commit _servicedata OR hellorust-0.1.1~git0.db340ad.obscpio OR hellorust.obsinfo</li> </ul> <pre data-lang="bash" style="background-color:#2b303b;color:#c0c5ce;" class="language-bash "><code class="language-bash" data-lang="bash"><span> </span><span style="color:#bf616a;">osc</span><span> ci </span></code></pre> <p>From here, you can use your packages from your own respository, or you can forward them to OpenSUSE Tumbleweed (via Factory). You likely need to polish and add extra parts to your package for it to be accepted into Factory, but this should at least make it easier for you to start!</p> <p>For more, see the <a href="https://en.opensuse.org/openSUSE:How_to_contribute_to_Factory">how to contribute to Factory</a> document. To submit to Leap, the package must be in Factory, then you can request it to be <a href="https://en.opensuse.org/openSUSE:Packaging_for_Leap">submitted to Leap</a> as well.</p> <p>Happy Contributing! 🦎🦀</p> Webauthn UserVerificationPolicy Curiosities Sat, 21 Nov 2020 00:00:00 +0000 Unknown https://fy.blackhats.net.au/blog/2020-11-21-webauthn-userverificationpolicy-curiosities/ https://fy.blackhats.net.au/blog/2020-11-21-webauthn-userverificationpolicy-curiosities/ <h1 id="webauthn-userverificationpolicy-curiosities">Webauthn UserVerificationPolicy Curiosities</h1> <p>Recently I received a <a href="https://github.com/kanidm/webauthn-rs/issues/32">pair</a> <a href="https://github.com/kanidm/webauthn-rs/issues/34">of</a> interesting bugs in <a href="https://github.com/kanidm/webauthn-rs/">Webauthn RS</a> where certain types of authenticators would not work in Firefox, but did work in Chromium. This confused me, and I couldn't reproduce the behaviour. So like any obsessed person I ordered myself one of the affected devices and waited for Australia Post to lose it, find it, lose it again, and then finally deliver the device 2 months later.</p> <p>In the meantime I swapped browsers from Firefox to Edge and started to notice some odd behaviour when logging into my corporate account - my yubikey began to ask me for my pin on every authentication, even though the key was registered to the corp servers <em>without</em> a pin. Yet the key kept working on Edge with a pin - and confusingly <em>without</em> a pin on Firefox.</p> <h2 id="some-background-on-webauthn">Some background on Webauthn</h2> <p>Before we dive into the issue, we need to understand some details about Webauthn. <a href="https://www.w3.org/TR/webauthn/">Webauthn</a> is a standard that allows a client (commonly a web browser) to cryptographically authenticate to a server (commonly a web site). Webauthn defines how different types of hardware cryptographic authenticators may communicate between the client and the server.</p> <p>An example of some types of authenticator devices are U2F tokens (yubikeys), TouchID (Apple Mac, iPhone, iPad), Trusted Platform Modules (Windows Hello) and many more. Webauthn has to account for differences in these hardware classes and how they communicate, but in the end each device performs a set of <a href="https://en.wikipedia.org/wiki/Public-key_cryptography">asymmetric cryptographic</a> (public/private key) operations.</p> <p>Webauthn defines the structures of how a client and server communicate to both register new authenticators and subsequently authenticate with those authenticators.</p> <p>For the first step of registration, the server provides a registration challenge to the client. The structure of this (which is important for later) looks like:</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>dictionary PublicKeyCredentialCreationOptions { </span><span> required PublicKeyCredentialRpEntity rp; </span><span> required PublicKeyCredentialUserEntity user; </span><span> </span><span> required BufferSource challenge; </span><span> required sequence&lt;PublicKeyCredentialParameters&gt; pubKeyCredParams; </span><span> </span><span> unsigned long timeout; </span><span> sequence&lt;PublicKeyCredentialDescriptor&gt; excludeCredentials = []; </span><span> AuthenticatorSelectionCriteria authenticatorSelection = { </span><span> AuthenticatorAttachment authenticatorAttachment; </span><span> boolean requireResidentKey = false; </span><span> UserVerificationRequirement userVerification = &quot;preferred&quot;; </span><span> }; </span><span> AttestationConveyancePreference attestation = &quot;none&quot;; </span><span> AuthenticationExtensionsClientInputs extensions; </span><span>}; </span></code></pre> <p>The client then takes this structure, and creates a number of hashes from it which the authenticator then signs. This signed data and options are returned to the server as a <a href="https://w3c.github.io/webauthn/#iface-pkcredential">PublicKeyCredential</a> containing an <a href="https://www.w3.org/TR/webauthn/#iface-authenticatorattestationresponse">Authenticator Attestation Response</a>.</p> <p>Next is authentication. The server sends a challenge to the client which has the structure:</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>dictionary PublicKeyCredentialRequestOptions { </span><span> required BufferSource challenge; </span><span> unsigned long timeout; </span><span> USVString rpId; </span><span> sequence&lt;PublicKeyCredentialDescriptor&gt; allowCredentials = []; </span><span> UserVerificationRequirement userVerification = &quot;preferred&quot;; </span><span> AuthenticationExtensionsClientInputs extensions; </span><span>}; </span></code></pre> <p>Again, the client takes this structure, takes a number of hashes and the authenticator signs this to prove it is the holder of the private key. The signed response is sent to the server as a PublicKeyCredential containing an <a href="https://www.w3.org/TR/webauthn/#iface-authenticatorassertionresponse">Authenticator Assertion Response</a>.</p> <p>Key to this discussion is the following field:</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>UserVerificationRequirement userVerification = &quot;preferred&quot;; </span></code></pre> <p>This is present in both PublicKeyCredentialRequestOptions and PublicKeyCredentialCreationOptions. This informs what level of interaction assurance should be provided during the signing process. These are discussed in <a href="https://pages.nist.gov/800-63-3/sp800-63b.html">NIST SP800-64b (5.1.7, 5.1.9)</a> (which is just an excellent document anyway, so read it).</p> <p>One aspect of these authenticators is that they must provide tamper proof evidence that a person is physically present and interacting with the device for the signature to proceed. This is important as it means that if someone is able to gain remote code execution on your system, they are unable to use your authenticator devices (even if it's part of the device, like touch id) as they are not physically present at the machine.</p> <p>Some authenticators are able to go beyond to strengthen this assurance, by verifying the identity of the person interacting with the authenticator. This means that the interaction <em>also</em> requires say a PIN (something you know), or a biometric (something you are). This allows the authenticator to assert not just that <em>someone</em> is present but that it is a specific person who is present.</p> <p>All authenticators are capable of asserting user presence but only some are capable of asserting user verification. This creates two classes of authenticators as defined by NIST SP800-64b.</p> <p>Single-Factor Cryptographic Devices (5.1.7) which only assert presence (the device becomes something you have) and Multi-Factor Cryptographic Devices (5.1.9) which assert the identity of the holder (something you have + something you know/are).</p> <p>Webauthn is able to request the use of a Single-Factor Device or Multi-Factor Device through it's UserVerificationRequirement option. The levels are:</p> <ul> <li>Discouraged - Only use Single-Factor Devices</li> <li>Required - Only use Multi-Factor Devices</li> <li>Preferred - Request Multi-Factor if possible, but allow Single-Factor devices.</li> </ul> <h2 id="back-to-the-mystery">Back to the mystery ...</h2> <p>When I initially saw these reports - of devices that did not work in Firefox but did in Chromium, and of devices asking for PINs on some browsers but not others -I was really confused. The breakthrough came as I was developing <a href="https://github.com/kanidm/webauthn-authenticator-rs">Webauthn Authenticator RS</a>. This is the client half of Webauthn, so that I could have the <a href="https://github.com/kanidm/kanidm">Kanidm</a> CLI tools use Webauthn for multi-factor authentication (MFA). In the process, I have been using the <a href="https://crates.io/crates/authenticator">authenticator</a> crate made by Mozilla and used by Firefox.</p> <p>The authenticator crate is what communicates to authenticators by NFC, Bluetooth, or USB. Due to the different types of devices, there are multiple different protocols involved. For U2F devices, the protocol is CTAP over USB. There are two versions of the CTAP protocol - CTAP1, and CTAP2.</p> <p>In the authenticator crate, only CTAP1 is supported. CTAP1 devices are unable to accept a PIN, so user verification must be performed internally to the device (such as a fingerprint reader built into the U2F device).</p> <p>Chromium, however, is able to use CTAP2 - CTAP2 <em>does</em> allow a PIN to be provided from the host machine to the device as a user verification method.</p> <h2 id="why-would-devices-fail-in-firefox">Why would devices fail in Firefox?</h2> <p>Once I had learnt this about CTAP1/CTAP2, I realised that my example code in Webauthn RS was hardcoding Required as the user verification level. Since Firefox can only use CTAP1, it was unable to use PINs to U2F devices, so they would not respond to the challenge. But on Chromium with CTAP2 they <em>are</em> able to have PINs so Required can be satisfied and the devices work.</p> <h2 id="okay-but-the-corp-account">Okay but the corp account?</h2> <p>This one is subtle. The corp identity system uses user verification of 'Preferred'. That meant that on Firefox, no PIN was requested since CTAP1 can't provide them, but on Edge/Chromium a PIN <em>can</em> be provided as they use CTAP2.</p> <p>What's more curious is that the same authenticator device is flipping between Single Factor and Multi Factor, with the same Private/Public Key pair just based on what protocol is used! So even though the 'Preferred' request can be satisfied on Chromium/Edge, it's not on Firefox. To further extend my confusion, the device was originally registered to the corp identity system in Firefox so it would have <em>not</em> had user verification available, but now that I use Edge it has <em>gained</em> this requirement during authentication.</p> <h2 id="that-seems-wrong">That seems ... wrong.</h2> <p>I agree. But Webauthn fully allows this. This is because user verification is a property of the <em>request/response</em> flow, not a property of the <em>device</em>.</p> <p>This creates some interesting side effects that become an opportunity for user confusion. (<em>I</em> was confused about what the behaviour was and I write a webauthn server and client library - imagine how other people feel ...).</p> <h2 id="devices-change-behaviour">Devices change behaviour</h2> <p>This means that during registration one policy can be requested (i.e. Required) but subsequently it may not be used (Preferred + Firefox + U2F, or Discouraged). Another example of a change in behaviour occurs when a device is used on Chromium with Preferred user verification is required, but when used on Firefox the device may <em>not</em> require verification. It also means that a site that implements Required can have devices that simply don't work in other browsers.</p> <p>Because this is changing behaviour it can confuse users. For examples:</p> <ul> <li>Why do I need a PIN now but not before?</li> <li>Why did I need a PIN before but not now?</li> <li>Why does my authenticator work on this computer but not on another?</li> </ul> <h2 id="preferred-becomes-discouraged">Preferred becomes Discouraged</h2> <p>This opens up a security risk where since Preferred &quot;attempts&quot; verification but allows it to not be present, a U2F device can be &quot;downgraded&quot; from Multi-Factor to Single-Factor by using it with CTAP1 instead of CTAP2. Since it's also per <em>request/response</em>, a compromised client could also tamper with the communication to the authenticator removing the requested userverification parameter silently and the server would allow it.</p> <p>This means that in reality, Preferred is policy and security wise equivalent to Discouraged, but with a more annoying UI/UX for users who have to conduct a verification that doesn't actually help identify them.</p> <p>Remember - if unspecified, 'Preferred' is the default user verification policy in Webauthn!</p> <h2 id="lock-out-abuse-vectors">Lock Out / Abuse Vectors</h2> <p>There is also a potential abuse vector here. Many devices such as U2F tokens perform a &quot;trust on first use&quot; for their PIN setup. This means that the first time a user verification is requested you configure the pin at that point in time.</p> <p>A potential abuse vector here is a token that is always used on Firefox, a malicious person could connect the device to Chromium, and setup the PIN without the knowledge of the owner. The owner could continue to use the device, and when Firefox eventually supports CTAP2, or they swap computer or browser, they would <em>not</em> know the PIN, and their token would effectively be unusable at that point. They would need to reset it, potentially causing them to be locked out from accounts, but more likely causing them to need to conduct a <em>lot</em> of password/credential resets.</p> <h2 id="unable-to-implement-authenticator-policy">Unable to implement Authenticator Policy</h2> <p>One of the greatest issues here though is that because user verification is part of the <em>request/response</em> flow and not <em>per device</em> attributes, authenticator policy, and mixed credentials are unable to exist in the current implementation of Webauthn.</p> <p>Consider a user who has enrolled say their laptop's U2F device + password, and their iPhone's touchID to a server. Both of these are Multi Factor credentials. The U2F is a Single Factor Device and becomes Multi-Factor in combination with the password. The iPhone touchID is a Multi-Factor Device on it's due to the biometric verification it is capable of.</p> <p>We <em>should</em> be able to have a website request webauthn and based on the device used we can flow to the next required step. If the device was the iPhone, we would be authenticated as we have authenticated a Multi Factor credentials. If we saw the U2F device we would then move to request the password since we have only received a Single Factor. However Webauthn is unable to express this authentication flow.</p> <p>If we requested Required, we would exclude the U2F device.</p> <p>If we requested Discouraged, we would exclude the iPhone.</p> <p>If we request Preferred, the U2F device could be used on a different browser with CTAP2, either:</p> <ul> <li>bypassing the password, since the device is now a self contained Multi-Factor; or</li> <li>the U2F device could prompt for the PIN needlessly and we progress to setting a password</li> </ul> <p>The request to an iPhone could be tampered with, preventing the verification occurring and turning it into a single factor device (presence only).</p> <p>Today, these mixed device scenarios can not exist in Webauthn. We are unable to create the policy around Single-Factor and Multi-Factor devices as defined by NIST because these require us to assert the verification requirements per credential, but Webauthn can not satisfy this.</p> <p>We would need to pre-ask the user <em>how</em> they want to authenticate on that device and then only send a Webauthn challenge that can satisfy the authentication policy we have decided on for those credentials.</p> <h2 id="how-to-fix-this">How to fix this</h2> <p>The solution here is to change PublicKeyCredentialDescriptor in the Webauthn standard to contain an optional UserVerificationRequirement field. This would allow a &quot;global&quot; default set by the server and then per-credential requirements to be defined. This would allow the user verification properties during registration to be associated to that credential, which can then be enforced by the server to guarantee the behaviour of a webauthn device. It would also allow the 'Preferred' option to have a valid and useful meaning during registration, where devices capable of verification can provide that or not, and then that verification boolean can be then transformed to a Discouraged or Required setting for that credential for future authentications.</p> <p>The second change would be to disallow 'Preferred' as a valid value in the &quot;global&quot; default during authentications. The new &quot;default&quot; global value should be 'Discouraged' and then only credentials that registered with verification would indicate that in their PublicKeyCredentialDescriptor.</p> <p>This would resolve the issues above by:</p> <ul> <li>Making the use of an authenticator consistent after registration. For example, authenticators registered with CTAP1 would stay 'Discouraged' even when used with CTAP2</li> <li>If PIN/Verification abuse occurred, the credentials registered on CTAP1 without verification would continue to be 'presence only' preventing the lockout</li> <li>Allowing the server to proceed with the authentication flow based on which credential authenticated and provide logic about further factors if needed.</li> <li>Allowing true Single Factor and Multi Factor device policies to be expressed in line with NIST SP800-63b, so users can have a mix of Single and Multi Factor devices associated with a single account.</li> </ul> <p>I have since opened <a href="https://github.com/w3c/webauthn/issues/1510">this issue</a> with the webauthn specification about this, but early comments seem to be highly focused on the current expression of the standard rather than the issues around the user experience and ability for identity systems to accurately express credential policy.</p> <p>In the meantime, I am going to make changes to Webauthn RS to help avoid some of these issues:</p> <ul> <li>Preferred will be renamed to Preferred_Is_Equivalent_To_Discouraged (it will still emit 'Preferred' in the JSON, this only changes the Rust API enum)</li> <li>Credential structures persisted by applications will contain the boolean of user-verification if it occurred during registration</li> <li>During an authentication, if the set of credentials contains inconsistent user-verification booleans, an error will be raised</li> <li>Authentication User Verification Policy is derived from the set of credentials having a consistent user-verification boolean</li> </ul> <p>While not perfect, it will mean that it's &quot;hard to hold it wrong&quot; with Webauthn RS.</p> <h2 id="acknowledgements">Acknowledgements</h2> <p>Thanks to both @Charcol0x89 and @JuxhinDB for reviewing this post.</p> Rust, SIMD and target-feature flags Fri, 20 Nov 2020 00:00:00 +0000 Unknown https://fy.blackhats.net.au/blog/2020-11-20-rust-and-target-feature-flags/ https://fy.blackhats.net.au/blog/2020-11-20-rust-and-target-feature-flags/ <h1 id="rust-simd-and-target-feature-flags">Rust, SIMD and target-feature flags</h1> <p>This year I've been working on <a href="https://github.com/kanidm/concread">concread</a> and one of the ways that I have improved it is through the use of <a href="https://github.com/rust-lang/packed_simd">packed_simd</a> for parallel key lookups in hashmaps. During testing I saw a ~10% speed up in Kanidm which heavily relies on concread, so great, pack it up, go home.</p> <h1 id="">...?</h1> <p>Or so I thought. Recently I was learning to use Ghidra with a friend, and as a thought exercise I wanted to see how Rust decompiled. I put the concread test suite into Ghidra and took a look. Looking at the version of concread with [simd_support]{.title-ref} enabled, I saw this in the disassembly (truncated for readability).</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>************************************************************** </span><span>* FUNCTION * </span><span>************************************************************** </span><span>Simd&lt;[packed_simd_2--masks--m64;8]&gt; __stdcall eq(Simd&lt;[p </span><span>... </span><span>100114510 55 PUSH RBP </span><span>100114511 48 89 e5 MOV RBP,RSP </span><span>100114514 48 83 e4 c0 AND RSP,-0x40 </span><span>100114518 48 81 ec SUB RSP,0x100 </span><span> 00 01 00 00 </span><span>10011451f 48 89 f8 MOV RAX,__return_storage_ptr__ </span><span>100114522 0f 28 06 MOVAPS XMM0,xmmword ptr [self-&gt;__0.__0] </span><span>... </span><span>100114540 66 0f 76 c4 PCMPEQD XMM0,XMM4 </span><span>100114544 66 0f 70 PSHUFD XMM4,XMM0,0xb1 </span><span> e0 b1 </span><span>100114549 66 0f db c4 PAND XMM0,XMM4 </span><span>... </span><span>100114574 0f 29 9c MOVAPS xmmword ptr [RSP + local_90],XMM3 </span><span> 24 b0 00 </span><span> 00 00 </span><span>1001145b4 48 89 7c MOV qword ptr [RSP + local_c8],__return_storage_pt </span><span> 24 78 </span><span>... </span><span>1001145be 0f 29 44 MOVAPS xmmword ptr [RSP + local_e0],XMM0 </span><span> 24 60 </span><span>... </span><span>1001145d2 48 8b 44 MOV RAX,qword ptr [RSP + local_c8] </span><span> 24 78 </span><span>1001145d7 0f 28 44 MOVAPS XMM0,xmmword ptr [RSP + local_e0] </span><span> 24 60 </span><span>1001145dc 0f 29 00 MOVAPS xmmword ptr [RAX],XMM0 </span><span>... </span><span>1001145ff 48 89 ec MOV RSP,RBP </span><span>100114602 5d POP RBP </span><span>100114603 c3 RET </span></code></pre> <p>Now, it's been a long time since I've had to look at x86_64 asm, so I saw this and went &quot;great, it's not using a loop, those aren't simple [TEST/JNZ]{.title-ref} instructions, they have a lot of letters, awesome it's using HW accel.</p> <h1 id="time-passes">Time passes ...</h1> <p>Coming back to this, I have been wondering how we could enable SIMD in concread at SUSE, since 389 Directory Server has just merged a change for 2.0.0 that uses concread as a cache. For this I needed to know what minimum CPU is supported at SUSE. After some chasing internally, knowing what we need I asked in the Rust Brisbane group about how you can define in [packed_simd]{.title-ref} to only emit instructions that work on a minimum CPU level rather than <em>my</em> cpu or the builder cpu.</p> <p>The response was &quot;but that's already how it works&quot;.</p> <p>I was helpfully directed to the <a href="https://rust-lang.github.io/packed_simd/perf-guide/target-feature/rustflags.html">packed_simd perf guide</a> which discusses the use of target features and target cpu. At that point I realised that for this whole time I've only been using the default:</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span># rustc --print cfg | grep -i target_feature </span><span>target_feature=&quot;fxsr&quot; </span><span>target_feature=&quot;sse&quot; </span><span>target_feature=&quot;sse2&quot; </span></code></pre> <p>The [PCMPEQD]{.title-ref} is from sse2, but my cpu is much newer and should support AVX and AVX2. Retesting this, I can see my CPU has much more:</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span># rustc --print cfg -C target-cpu=native | grep -i target_feature </span><span>target_feature=&quot;aes&quot; </span><span>target_feature=&quot;avx&quot; </span><span>target_feature=&quot;avx2&quot; </span><span>target_feature=&quot;bmi1&quot; </span><span>target_feature=&quot;bmi2&quot; </span><span>target_feature=&quot;fma&quot; </span><span>target_feature=&quot;fxsr&quot; </span><span>target_feature=&quot;lzcnt&quot; </span><span>target_feature=&quot;pclmulqdq&quot; </span><span>target_feature=&quot;popcnt&quot; </span><span>target_feature=&quot;rdrand&quot; </span><span>target_feature=&quot;rdseed&quot; </span><span>target_feature=&quot;sse&quot; </span><span>target_feature=&quot;sse2&quot; </span><span>target_feature=&quot;sse3&quot; </span><span>target_feature=&quot;sse4.1&quot; </span><span>target_feature=&quot;sse4.2&quot; </span><span>target_feature=&quot;ssse3&quot; </span><span>target_feature=&quot;xsave&quot; </span><span>target_feature=&quot;xsavec&quot; </span><span>target_feature=&quot;xsaveopt&quot; </span><span>target_feature=&quot;xsaves&quot; </span></code></pre> <p>All this time, I haven't been using my native features!</p> <p>For local builds now, I have .cargo/config set with:</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>[build] </span><span>rustflags = &quot;-C target-cpu=native&quot; </span></code></pre> <p>I recompiled concread and I now see in Ghidra:</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>00198960 55 PUSH RBP </span><span>00198961 48 89 e5 MOV RBP,RSP </span><span>00198964 48 83 e4 c0 AND RSP,-0x40 </span><span>00198968 48 81 ec SUB RSP,0x100 </span><span> 00 01 00 00 </span><span>0019896f 48 89 f8 MOV RAX,__return_storage_ptr__ </span><span>00198972 c5 fc 28 06 VMOVAPS YMM0,ymmword ptr [self-&gt;__0.__0] </span><span>00198976 c5 fc 28 VMOVAPS YMM1,ymmword ptr [RSI + self-&gt;__0.__4] </span><span> 4e 20 </span><span>0019897b c5 fc 28 12 VMOVAPS YMM2,ymmword ptr [other-&gt;__0.__0] </span><span>0019897f c5 fc 28 VMOVAPS YMM3,ymmword ptr [RDX + other-&gt;__0.__4] </span><span> 5a 20 </span><span>00198984 c4 e2 7d VPCMPEQQ YMM0,YMM0,YMM2 </span><span> 29 c2 </span><span>00198989 c4 e2 75 VPCMPEQQ YMM1,YMM1,YMM3 </span><span> 29 cb </span><span>0019898e c5 fc 29 VMOVAPS ymmword ptr [RSP + local_a0[0]],YMM1 </span><span> 8c 24 a0 </span><span> 00 00 00 </span><span>... </span><span>001989e7 48 89 ec MOV RSP,RBP </span><span>001989ea 5d POP RBP </span><span>001989eb c5 f8 77 VZEROUPPER </span><span>001989ee c3 RET </span></code></pre> <p>[VPCMPEQQ]{.title-ref} is the AVX2 compare instruction (You can tell it's AVX2 due to the register YMM, AVX uses XMM). Which means now I'm getting the SIMD comparisons I wanted!</p> <p>These can be enabled with [RUSTFLAGS='-C target-feature=+avx2,+avx']{.title-ref} for selected builds, or in your .cargo/config. It may be a good idea for just local development to do [target-cpu=native]{.title-ref}.</p> Deploying sccache on SUSE Thu, 19 Nov 2020 00:00:00 +0000 Unknown https://fy.blackhats.net.au/blog/2020-11-19-deploying-sccache-on-suse/ https://fy.blackhats.net.au/blog/2020-11-19-deploying-sccache-on-suse/ <h1 id="deploying-sccache-on-suse">Deploying sccache on SUSE</h1> <p>sccache is a ccache/icecc-like tool from Mozilla, which in addition to working with C and C++, is also able to help with Rust builds.</p> <h2 id="adding-the-repo">Adding the Repo</h2> <p>A submission to Factory (tumbleweed) has been made, so check if you can install from zypper:</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>zypper install sccache </span></code></pre> <p>If not, sccache is still part of <a href="https://build.opensuse.org/package/show/devel:tools:building/sccache">devel:tools:building</a> so you will need to add the repo to use sccache.</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>zypper ar -f obs://devel:tools:building devel:tools:building </span><span>zypper install sccache </span></code></pre> <p>It's also important you <em>do not</em> have ccache installed. ccache intercepts the gcc command so you end up &quot;double caching&quot; potentially.</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>zypper rm ccache </span></code></pre> <h2 id="single-host">Single Host</h2> <p>To use sccache on your host, you need to set the following environment variables:</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>export RUSTC_WRAPPER=sccache </span><span>export CC=&quot;sccache /usr/bin/gcc&quot; </span><span>export CXX=&quot;sccache /usr/bin/g++&quot; </span><span># Optional: This can improve rust caching </span><span># export CARGO_INCREMENTAL=false </span></code></pre> <p>This will allow sccache to wrap your compiler commands. You can show your current sccache status with:</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>sccache -s </span></code></pre> <p>There is more information about using cloud/remote storage for the cache on the sccache <a href="https://github.com/mozilla/sccache/blob/master/README.md">project site</a></p> <h2 id="distributed-compiliation">Distributed Compiliation</h2> <p>sccache is also capable of distributed compilation, where a number of builder servers can compile items and return the artificats to your machine. This can save you time by allowing compilation over a cluster, using a faster remote builder, or just to keep your laptop cool.</p> <p>Three components are needed to make this work. A scheduler that coordinates the activities, one or more builders that provide their CPU, and a client that submits compilation jobs.</p> <p>The sccache package contains the required elements for all three parts.</p> <p>Note that the client does <em>not</em> need to be the same version of SUSE or even the same distro as the scheduler or builder. This is because the client is able to bundle and submit it's toolchains to the workers on the fly. Neat! sccache is capacble of also compiling for macos and windows, but in these cases the toolchains can-not be submitted on the fly and requires extra <a href="https://github.com/mozilla/sccache/blob/master/docs/DistributedQuickstart.md">work to configure.</a></p> <h2 id="scheduler">Scheduler</h2> <p>The scheduler is configured with [/etc/sccache/scheduler.conf]{.title-ref}. You need to define the listening ip, client auth, and server (builder) auth methods. The example configuration is well commented to help with this:</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span># The socket address the scheduler will listen on. It&#39;s strongly recommended </span><span># to listen on localhost and put a HTTPS server in front of it. </span><span>public_addr = &quot;127.0.0.1:10600&quot; </span><span># public_addr = &quot;[::1]:10600&quot; </span><span> </span><span>[client_auth] </span><span># This is how a client will authenticate to the scheduler. </span><span># # sccache-dist auth generate-shared-token --help </span><span>type = &quot;token&quot; </span><span>token = &quot;token here&quot; </span><span># </span><span># type = &quot;jwt_hs256&quot; </span><span># secret_key = &quot;&quot; </span><span> </span><span>[server_auth] </span><span># sccache-dist auth --help </span><span># To generate the secret_key: </span><span># # sccache-dist auth generate-jwt-hs256-key </span><span># To generate a key for a builder, use the command: </span><span># # sccache-dist auth generate-jwt-hs256-server-token --config /etc/sccache/scheduler.conf --secret-key &quot;...&quot; --server &quot;builderip:builderport&quot; </span><span>type = &quot;jwt_hs256&quot; </span><span>secret_key = &quot;my secret key&quot; </span></code></pre> <p>You can start the scheduler with:</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>systemctl start sccache-dist-scheduler.service </span></code></pre> <p>If you have issues you can increase logging verbosity with:</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span># systemctl edit sccache-dist-scheduler.service </span><span>[Service] </span><span>Environment=&quot;RUST_LOG=sccache=trace&quot; </span></code></pre> <h2 id="builder">Builder</h2> <p>Similar to the scheduler, the builder is configured with [/etc/sccache/builder.conf]{.title-ref}. Most of the defaults should be left &quot;as is&quot; but you will need to add the token generated from the comments in [scheduler.conf - server_auth]{.title-ref}.</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span># This is where client toolchains will be stored. </span><span># You should not need to change this as it is configured to work with systemd. </span><span>cache_dir = &quot;/var/cache/sccache-builder/toolchains&quot; </span><span># The maximum size of the toolchain cache, in bytes. </span><span># If unspecified the default is 10GB. </span><span># toolchain_cache_size = 10737418240 </span><span># A public IP address and port that clients will use to connect to this builder. </span><span>public_addr = &quot;127.0.0.1:10501&quot; </span><span># public_addr = &quot;[::1]:10501&quot; </span><span> </span><span># The URL used to connect to the scheduler (should use https, given an ideal </span><span># setup of a HTTPS server in front of the scheduler) </span><span>scheduler_url = &quot;https://127.0.0.1:10600&quot; </span><span> </span><span>[builder] </span><span>type = &quot;overlay&quot; </span><span># The directory under which a sandboxed filesystem will be created for builds. </span><span># You should not need to change this as it is configured to work with systemd. </span><span>build_dir = &quot;/var/cache/sccache-builder/tmp&quot; </span><span># The path to the bubblewrap version 0.3.0+ `bwrap` binary. </span><span># You should not need to change this as it is configured for a default SUSE install. </span><span>bwrap_path = &quot;/usr/bin/bwrap&quot; </span><span> </span><span>[scheduler_auth] </span><span>type = &quot;jwt_token&quot; </span><span># This will be generated by the `generate-jwt-hs256-server-token` command or </span><span># provided by an administrator of the sccache cluster. See /etc/sccache/scheduler.conf </span><span>token = &quot;token goes here&quot; </span></code></pre> <p>Again, you can start the builder with:</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>systemctl start sccache-dist-builder.service </span></code></pre> <p>If you have issues you can increase logging verbosity with:</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span># systemctl edit sccache-dist-builder.service </span><span>[Service] </span><span>Environment=&quot;RUST_LOG=sccache=trace&quot; </span></code></pre> <p>You can configure many hosts as builders, and compilation jobs will be distributed amongst them.</p> <h2 id="client">Client</h2> <p>The client is the part that submits compilation work. You need to configure your machine the same as single host with regard to the environment variables.</p> <p>Additionally you need to configure the file [~/.config/sccache/config]{.title-ref}. An example of this can be found in [/etc/sccache/client.example]{.title-ref}.</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>[dist] </span><span># The URL used to connect to the scheduler (should use https, given an ideal </span><span># setup of a HTTPS server in front of the scheduler) </span><span>scheduler_url = &quot;http://x.x.x.x:10600&quot; </span><span># Used for mapping local toolchains to remote cross-compile toolchains. Empty in </span><span># this example where the client and build server are both Linux. </span><span>toolchains = [] </span><span># Size of the local toolchain cache, in bytes (5GB here, 10GB if unspecified). </span><span># toolchain_cache_size = 5368709120 </span><span> </span><span>cache_dir = &quot;/tmp/toolchains&quot; </span><span> </span><span>[dist.auth] </span><span>type = &quot;token&quot; </span><span># This should match the `client_auth` section of the scheduler config. </span><span>token = &quot;&quot; </span></code></pre> <p>You can check the status with:</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>sccache --stop-server </span><span>sccache --dist-status </span></code></pre> <p>If you have issues, you can increase the logging with:</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>sccache --stop-server </span><span>SCCACHE_NO_DAEMON=1 RUST_LOG=sccache=trace sccache --dist-status </span></code></pre> <p>Then begin a compilation job and you will get the extra logging. To undo this, run:</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>sccache --stop-server </span><span>sccache --dist-status </span></code></pre> <p>In addition, sccache even in distributed mode can still use cloud or remote storage for items, using it's cache first, and the distributed complitation second. Anything that can't be remotely complied will be run locally.</p> <h2 id="verifying">Verifying</h2> <p>If you compile something from your client, you should see messages like this appear in journald in the builder/scheduler machine:</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>INFO 2020-11-19T22:23:46Z: sccache_dist: Job 140 created and will be assigned to server ServerId(V4(x.x.x.x:10501)) </span><span>INFO 2020-11-19T22:23:46Z: sccache_dist: Job 140 successfully assigned and saved with state Ready </span><span>INFO 2020-11-19T22:23:46Z: sccache_dist: Job 140 updated state to Started </span><span>INFO 2020-11-19T22:23:46Z: sccache_dist: Job 140 updated state to Complete </span></code></pre> How a Search Query is Processed in Kanidm Tue, 01 Sep 2020 00:00:00 +0000 Unknown https://fy.blackhats.net.au/blog/2020-09-01-how-a-search-query-is-processed-in-kanidm/ https://fy.blackhats.net.au/blog/2020-09-01-how-a-search-query-is-processed-in-kanidm/ <h1 id="how-a-search-query-is-processed-in-kanidm">How a Search Query is Processed in Kanidm</h1> <p>Databases from postgres to sqlite, mongodb, and even LDAP all need to take a query and turn that into a meaningful result set. This process can often seem like magic, especially when you consider an LDAP server is able to process thousands of parallel queries, with a database spanning millions of entries and still can return results in less than a millisecond. Even more impressive is that every one of these databases can be expected to return the correct result, every time. This level of performance, correctness and precision is an astounding feat of engineering, but is rooted in a simple set of design patterns.</p> <h2 id="disclaimer">Disclaimer</h2> <p>This will be a very long post. You may want to set aside some time for it :)</p> <p>This post will discuss how <a href="https://github.com/kanidm/kanidm">Kanidm</a> processes queries. This means that some implementation specifics are specific to the Kanidm project. However conceptually this is very close to the operation of LDAP servers (389-ds, samba 4, openldap) and MongoDB, and certainly there are still many overlaps and similarities to SQLite and Postgres. At the least, I hope it gives you some foundation to research the specifics behaviours you chosen database.</p> <p>This post does NOT discuss how creation or modification paths operate. That is likely worthy of a post of it's own. Saying this, search relies heavily on correct function of the write paths, and they are always intertwined.</p> <p>The referenced code and links relate to commit <a href="https://github.com/kanidm/kanidm/tree/dbfe87e675beac7fd931a445fd80cf439c2c6e61">dbfe87e</a> from 2020-08-24. The project may have changed since this point, so it's best if you can look at the latest commits in the tree if possible.</p> <h2 id="introduction">Introduction</h2> <p>Kanidm uses a structured document store model, similar to LDAP or MongoDB. You can consider entries to be like a JSON document. For example,</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>{ </span><span> &quot;class&quot;: [ </span><span> &quot;object&quot;, </span><span> &quot;memberof&quot;, </span><span> &quot;account&quot;, </span><span> &quot;posixaccount&quot; </span><span> ], </span><span> &quot;displayname&quot;: [ </span><span> &quot;William&quot; </span><span> ], </span><span> &quot;gidnumber&quot;: [ </span><span> &quot;1000&quot; </span><span> ], </span><span> &quot;loginshell&quot;: [ </span><span> &quot;/bin/zsh&quot; </span><span> ], </span><span> &quot;name&quot;: [ </span><span> &quot;william&quot; </span><span> ], </span><span> &quot;uuid&quot;: [ </span><span> &quot;5e01622e-740a-4bea-b694-e952653252b4&quot; </span><span> ], </span><span> &quot;memberof&quot;: [ </span><span> &quot;admins&quot;, </span><span> &quot;users&quot;, </span><span> &quot;radius&quot; </span><span> ], </span><span> &quot;ssh_publickey&quot;: [ </span><span> { </span><span> &quot;tag&quot;: &quot;laptop&quot;, </span><span> &quot;key&quot;: &quot;....&quot; </span><span> } </span><span> ] </span><span>} </span></code></pre> <p>Something of note here is that an entry has many attributes, and those attributes can consist of one or more values. values themself can be structured such as the ssh_publickey value which has a tag and the public key, or the uuid which enforces uuid syntax.</p> <h2 id="filters-queries">Filters / Queries</h2> <p>During a search we want to find entries that match specific attribute value assertions or attribute assertions. We also want to be able to use logic to provide complex conditions or logic in how we perform the search. We could consider the search in terms of SQL such as:</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>select from entries where name = william and class = account; </span></code></pre> <p>Or in LDAP syntax</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>(&amp;(objectClass=account)(name=william)) </span></code></pre> <p>In Kanidm JSON (which admitedly, is a bit rough, we don't expect people to use this much!)</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>{ &quot;and&quot;: [{&quot;eq&quot;: [&quot;class&quot;, &quot;account&quot;]}, {&quot;eq&quot;: [&quot;name&quot;: &quot;william&quot;]} ]} </span></code></pre> <p>Regardless of how we structure these, they are the same query. We want to find entries where the property of class=account and name=william hold true. There are many other types of logic we could apply (especially true for sql), but in Kanidm we support the following <a href="https://github.com/kanidm/kanidm/blob/dbfe87e675beac7fd931a445fd80cf439c2c6e61/kanidm_proto/src/v1.rs#L305">proto(col) filters</a></p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>pub enum Filter { </span><span> Eq(String, String), </span><span> Sub(String, String), </span><span> Pres(String), </span><span> Or(Vec&lt;Filter&gt;), </span><span> And(Vec&lt;Filter&gt;), </span><span> AndNot(Box&lt;Filter&gt;), </span><span> SelfUUID, </span><span>} </span></code></pre> <p>These represent:</p> <ul> <li>Eq(uality) - an attribute of name, has at least one value matching the term</li> <li>Sub(string) - an attribute of name, has at least one value matching the substring term</li> <li>Pres(ence) - an attribute of name, regardless of value exists on the entry</li> <li>Or - One or more of the nested conditions must evaluate to true</li> <li>And - All nested conditions must be true, or the and returns false</li> <li>AndNot - Within an And query, the inner term must not be true relative to the related and term</li> <li>SelfUUID - A dynamic Eq(uality) where the authenticated user's UUID is added. Essentially, this substitutes to &quot;eq (uuid, selfuuid)&quot;</li> </ul> <p>Comparing to the previous example entry, we can see that [{ &quot;and&quot;: [{&quot;eq&quot;: [&quot;class&quot;, &quot;account&quot;]}, {&quot;eq&quot;: [&quot;name&quot;: &quot;william&quot;]} ]}]{.title-ref} would be true, where [{ &quot;eq&quot;: [&quot;name&quot;: &quot;claire&quot;]}]{.title-ref} would be false as no matching name attribute-value exists on the entry.</p> <h2 id="recieving-the-query">Recieving the Query</h2> <p>There are multiple ways that a query could find it's way into Kanidm. It may be submitted from the raw search api, it could be generated from a REST endpoint request, it may be translated via the LDAP compatability. The most important part is that it is then recieved by a worker thread in the query server. For this discussion we'll assume we recieved a raw search via the front end.</p> <p><a href="https://github.com/kanidm/kanidm/blob/dbfe87e675beac7fd931a445fd80cf439c2c6e61/kanidmd/src/lib/actors/v1_read.rs#L245">handle_search</a> is the entry point of a worker thread to process a search operation. The first thing we do is begin a read transaction over the various elements of the database we need.</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>fn handle(&amp;mut self, msg: SearchMessage, _: &amp;mut Self::Context) -&gt; Self::Result { </span><span>let mut audit = AuditScope::new(&quot;search&quot;, msg.eventid, self.log_level); </span><span>let res = lperf_op_segment!(&amp;mut audit, &quot;actors::v1_read::handle&lt;SearchMessage&gt;&quot;, || { </span><span> // Begin a read </span><span> let qs_read = self.qs.read(); </span></code></pre> <p>The call to <a href="https://github.com/kanidm/kanidm/blob/dbfe87e675beac7fd931a445fd80cf439c2c6e61/kanidmd/src/lib/server.rs#L732">qs.read</a> takes three transactions - the backend, the schema cache and the access control cache.</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>pub fn read(&amp;self) -&gt; QueryServerReadTransaction { </span><span> QueryServerReadTransaction { </span><span> be_txn: self.be.read(), </span><span> schema: self.schema.read(), </span><span> accesscontrols: self.accesscontrols.read(), </span><span> } </span><span>} </span></code></pre> <p>The <a href="https://github.com/kanidm/kanidm/blob/dbfe87e675beac7fd931a445fd80cf439c2c6e61/kanidmd/src/lib/be/mod.rs#L1348">backend read</a> takes two transactions internally - the database layers, and the indexing metadata cache.</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>pub fn read(&amp;self) -&gt; BackendReadTransaction { </span><span> BackendReadTransaction { </span><span> idlayer: UnsafeCell::new(self.idlayer.read()), </span><span> idxmeta: self.idxmeta.read(), </span><span> } </span><span>} </span></code></pre> <p>Once complete, we can now transform the submitted request, into an internal event. By structuring all requests as event, we standarise all operations to a subset of operations, and we ensure that that all resources required are available in the event. The <a href="https://github.com/kanidm/kanidm/blob/dbfe87e675beac7fd931a445fd80cf439c2c6e61/kanidmd/src/lib/event.rs#L255">search event</a> as processed stores an event origin aka the identiy of the event origin. The search query is stored in the [filter]{.title-ref} attribute, and the original query is stored in the [filter_orig]{.title-ref}. There is a reason for this duplication.</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>pub fn from_message( </span><span> audit: &amp;mut AuditScope, </span><span> msg: SearchMessage, </span><span> qs: &amp;QueryServerReadTransaction, </span><span>) -&gt; Result&lt;Self, OperationError&gt; { </span><span> let event = Event::from_ro_uat(audit, qs, msg.uat.as_ref())?; </span><span> let f = Filter::from_ro(audit, &amp;event, &amp;msg.req.filter, qs)?; </span><span> // We do need to do this twice to account for the ignore_hidden </span><span> // changes. </span><span> let filter = f </span><span> .clone() </span><span> .into_ignore_hidden() </span><span> .validate(qs.get_schema()) </span><span> .map_err(OperationError::SchemaViolation)?; </span><span> let filter_orig = f </span><span> .validate(qs.get_schema()) </span><span> .map_err(OperationError::SchemaViolation)?; </span><span> Ok(SearchEvent { </span><span> event, </span><span> filter, </span><span> filter_orig, </span><span> // We can&#39;t get this from the SearchMessage because it&#39;s annoying with the </span><span> // current macro design. </span><span> attrs: None, </span><span> }) </span><span>} </span></code></pre> <p>As [filter]{.title-ref} is processed it is transformed by the server to change it's semantics. This is due to the call to <a href="https://github.com/kanidm/kanidm/blob/dbfe87e675beac7fd931a445fd80cf439c2c6e61/kanidmd/src/lib/filter.rs#L504">into_ignore_hidden</a>. This function adds a wrapping layer to the outside of the query that hides certain classes of entries from view unless explicitly requested. In the case of kanidm this transformation is to add:</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>{ &quot;and&quot;: [ </span><span> { &quot;andnot&quot; : { &quot;or&quot; [ </span><span> {&quot;eq&quot;: [&quot;class&quot;, &quot;tombstone&quot;]}, </span><span> {&quot;eq&quot;: [&quot;class&quot;, &quot;recycled&quot;]} </span><span> }]}, </span><span> &lt;original query&gt; </span><span>]} </span></code></pre> <p>This prevents the display of deleted (recycle bin) entries, and the display of tombstones - marker entries representing that an entry with this UUID once existed in this location. These tombstones are part of the (future) eventually consistent replication machinery to allow deletes to be processed.</p> <p>This is why [filter_orig]{.title-ref} is stored. We require a copy of the &quot;query as intended by the caller&quot; for the purpose of checking access controls later. A user may not have access to the attribute &quot;class&quot; which would mean that the addition of the [into_ignore_hidden]{.title-ref} could cause them to not have any results at all. We should not penalise the user for something they didn't ask for!</p> <p>After the query is transformed, we now <a href="https://github.com/kanidm/kanidm/blob/dbfe87e675beac7fd931a445fd80cf439c2c6e61/kanidmd/src/lib/event.rs#L277">validate</a> it's content. This validation ensures that queries contain only attributes that truly exist in schema, and that their representation in the query is sound. This prevents a number of security issues related to denial of service or possible information disclosures. The query has every attribute-value <a href="https://github.com/kanidm/kanidm/blob/dbfe87e675beac7fd931a445fd80cf439c2c6e61/kanidmd/src/lib/filter.rs#L545">compared</a> to the schema to ensure that they exist and are correct syntax types.</p> <h2 id="start-processing-the-query">Start Processing the Query</h2> <p>Now that the search event has been created and we know that is is valid within a set of rules, we can submit it to the <a href="https://github.com/kanidm/kanidm/blob/dbfe87e675beac7fd931a445fd80cf439c2c6e61/kanidmd/src/lib/actors/v1_read.rs#L265">search_ext(ernal)</a> interface of the query server. Because everything we need is contained in the search event we are able to process the query from this point. Search external is a wrapper to the internal search, where <a href="https://github.com/kanidm/kanidm/blob/dbfe87e675beac7fd931a445fd80cf439c2c6e61/kanidmd/src/lib/server.rs#L90">search_ext</a> is able to wrap and apply access controls to the results from the operation.</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>fn search_ext( </span><span> &amp;self, </span><span> au: &amp;mut AuditScope, </span><span> se: &amp;SearchEvent, </span><span>) -&gt; Result&lt;Vec&lt;Entry&lt;EntryReduced, EntryCommitted&gt;&gt;, OperationError&gt; { </span><span> lperf_segment!(au, &quot;server::search_ext&quot;, || { </span><span> /* </span><span> * This just wraps search, but it&#39;s for the external interface </span><span> * so as a result it also reduces the entry set&#39;s attributes at </span><span> * the end. </span><span> */ </span><span> let entries = self.search(au, se)?; </span><span> </span><span> let access = self.get_accesscontrols(); </span><span> access </span><span> .search_filter_entry_attributes(au, se, entries) </span><span> .map_err(|e| { </span><span> // Log and fail if something went wrong. </span><span> ladmin_error!(au, &quot;Failed to filter entry attributes {:?}&quot;, e); </span><span> e </span><span> }) </span><span> // This now returns the reduced vec. </span><span> }) </span><span>} </span></code></pre> <p>The <a href="https://github.com/kanidm/kanidm/blob/dbfe87e675beac7fd931a445fd80cf439c2c6e61/kanidmd/src/lib/server.rs#L90">internal search</a> function is now called, and we begin to prepare for the backend to handle the query.</p> <p>We have a final transformation we must apply to the query that we intend to pass to the backend. We must attach metadata to the query elements so that we can perform informed optimisation of the query.</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>let be_txn = self.get_be_txn(); </span><span>let idxmeta = be_txn.get_idxmeta_ref(); </span><span>// Now resolve all references and indexes. </span><span>let vfr = lperf_trace_segment!(au, &quot;server::search&lt;filter_resolve&gt;&quot;, || { </span><span> se.filter.resolve(&amp;se.event, Some(idxmeta)) </span><span>}) </span></code></pre> <p>This is done by retreiving indexing metadata from the backend, which defines which attributes and types of indexes exist. This indexing metadata is passed to the filter to <a href="https://github.com/kanidm/kanidm/blob/dbfe87e675beac7fd931a445fd80cf439c2c6e61/kanidmd/src/lib/filter.rs#L504">be resolved</a>. In the case of tests we may not pass index metadata, which is why filter resolve accounts for the possibility of idxmeta being None. The filter elements are transformed, for example we change <a href="https://github.com/kanidm/kanidm/blob/dbfe87e675beac7fd931a445fd80cf439c2c6e61/kanidmd/src/lib/filter.rs#L973">eq to have a boolean</a> associated if the attribute is indexed. In our example this would change the query:</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>{ &quot;and&quot;: [ </span><span> { &quot;andnot&quot; : { &quot;or&quot; [ </span><span> {&quot;eq&quot;: [&quot;class&quot;, &quot;tombstone&quot;]}, </span><span> {&quot;eq&quot;: [&quot;class&quot;, &quot;recycled&quot;]} </span><span> }]}, </span><span> { &quot;and&quot;: [ </span><span> {&quot;eq&quot;: [&quot;class&quot;, &quot;account&quot;]}, </span><span> {&quot;eq&quot;: [&quot;name&quot;: &quot;william&quot;]} </span><span> ]} </span><span>]} </span></code></pre> <p>To</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>{ &quot;and&quot;: [ </span><span> { &quot;andnot&quot; : { &quot;or&quot; [ </span><span> {&quot;eq&quot;: [&quot;class&quot;, &quot;tombstone&quot;, true]}, </span><span> {&quot;eq&quot;: [&quot;class&quot;, &quot;recycled&quot;, true]} </span><span> }]}, </span><span> { &quot;and&quot;: [ </span><span> {&quot;eq&quot;: [&quot;class&quot;, &quot;account&quot;, true]}, </span><span> {&quot;eq&quot;: [&quot;name&quot;: &quot;william&quot;, true]} </span><span> ]} </span><span>]} </span></code></pre> <p>With this metadata associated to the query, we can now submit it to the backend for processing.</p> <h2 id="backend-processing">Backend Processing</h2> <p>We are now in a position where the backend can begin to do work to actually process the query. The first step of the <a href="https://github.com/kanidm/kanidm/blob/dbfe87e675beac7fd931a445fd80cf439c2c6e61/kanidmd/src/lib/be/mod.rs#L474">backend search</a> function is to perform the final optimisation of the filter.</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>fn search( </span><span> &amp;self, </span><span> au: &amp;mut AuditScope, </span><span> erl: &amp;EventLimits, </span><span> filt: &amp;Filter&lt;FilterValidResolved&gt;, </span><span>) -&gt; Result&lt;Vec&lt;Entry&lt;EntrySealed, EntryCommitted&gt;&gt;, OperationError&gt; { </span><span> lperf_trace_segment!(au, &quot;be::search&quot;, || { </span><span> // Do a final optimise of the filter </span><span> let filt = </span><span> lperf_trace_segment!(au, &quot;be::search&lt;filt::optimise&gt;&quot;, || { filt.optimise() }); </span></code></pre> <p>Query optimisation is critical to make searches fast. In Kanidm it relies on a specific behaviour of the indexing application process. I will highlight that step shortly.</p> <p>For now, the way query optimisation works is by sorting and folding terms in the query. This is because there are a number of logical equivalences, but also that due to the associated metadata and experience we know that some terms may be better in different areas. Optimisation relies on a <a href="https://github.com/kanidm/kanidm/blob/dbfe87e675beac7fd931a445fd80cf439c2c6e61/kanidmd/src/lib/filter.rs#L1088">sorting function</a> that will rearrange terms as needed.</p> <p>An example is that a nested [and]{.title-ref} term, can be folded to the parent because logically an [and]{.title-ref} inside and [and]{.title-ref} is the same. Similar for [or]{.title-ref} inside [or]{.title-ref}.</p> <p>Within the [and]{.title-ref} term, we can then rearrange the terms, because the order of the terms does not matter in an [and]{.title-ref} or [or]{.title-ref}, only that the other logical elements hold true. We sort indexed equality terms first because we know that they are always going to resolve &quot;faster&quot; than the nested [andnot]{.title-ref} term.</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>{ &quot;and&quot;: [ </span><span> {&quot;eq&quot;: [&quot;class&quot;, &quot;account&quot;, true]}, </span><span> {&quot;eq&quot;: [&quot;name&quot;: &quot;william&quot;, true]}, </span><span> { &quot;andnot&quot; : { &quot;or&quot; [ </span><span> {&quot;eq&quot;: [&quot;class&quot;, &quot;tombstone&quot;, true]}, </span><span> {&quot;eq&quot;: [&quot;class&quot;, &quot;recycled&quot;, true]} </span><span> }]} </span><span>]} </span></code></pre> <p>In the future, an improvement here is to put name before class, which will happen as part of the issue <a href="https://github.com/kanidm/kanidm/issues/238">#238</a> which allows us to work out which indexes are going to yield the best information content. So we can sort them first in the query.</p> <p>Finally, we are at the point where we can begin to actually load some data! 🎉</p> <p>The filter is submitted to <a href="https://github.com/kanidm/kanidm/blob/dbfe87e675beac7fd931a445fd80cf439c2c6e61/kanidmd/src/lib/be/mod.rs#L109">filter2idl</a>. To understand this function, we need to understand how indexes and entries are stored.</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>let (idl, fplan) = lperf_trace_segment!(au, &quot;be::search -&gt; filter2idl&quot;, || { </span><span> self.filter2idl(au, filt.to_inner(), FILTER_SEARCH_TEST_THRESHOLD) </span><span>})?; </span></code></pre> <p>All databases at the lowest levels are built on collections of key-value stores. That keyvalue store may be a in memory tree or hashmap, or an on disk tree. Some common stores are BDB, LMDB, SLED. In Kanidm we use SQLite as a key-value store, through tables that only contain two columns. The intent is to swap to SLED in the future once it gains transactions over a collection of trees, and that trees can be created/removed in transactions.</p> <p>The primary storage of all entries is in the table <a href="https://github.com/kanidm/kanidm/blob/master/kanidmd/src/lib/be/idl_sqlite.rs#L1134">id2entry</a> which has an id column (the key) and stores serialised entries in the data column.</p> <p>Indexes are stored in a collection of their own tables, named in the scheme &quot;idx_&lt;type&gt;_&lt;attr&gt;&quot;. For example, &quot;idx_eq_name&quot; or &quot;idx_pres_class&quot;. These are stored as two columns, where the &quot;key&quot; column is a precomputed result of a value in the entry, and the &quot;value&quot; is a set of integer ID's related to the entries that contain the relevant match.</p> <p>As a bit more of a graphic example, you can imagine these as:</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>id2entry </span><span>| id | data | </span><span>| 1 | { &quot;name&quot;: &quot;william&quot;, ... } </span><span>| 2 | { &quot;name&quot;: &quot;claire&quot;, ... } </span><span> </span><span>idx_eq_name </span><span>| key | </span><span>| william | [1, ] </span><span>| claire | [2, ] </span><span> </span><span>idm_eq_class </span><span>| account | [1, 2, ... ] </span></code></pre> <p>As these are key-value stores, they are able to be cached through an in-memory key value store to speed up the process. Initially, we'll assume these are not cache.</p> <h2 id="filter2idl">filter2idl</h2> <p>Back to [filter2idl]{.title-ref}. The query begins by processing the outer [and]{.title-ref} term. As the [and]{.title-ref} progresses inner elements are <a href="https://github.com/kanidm/kanidm/blob/dbfe87e675beac7fd931a445fd80cf439c2c6e61/kanidmd/src/lib/be/mod.rs#L229">iterated over</a> and then recursively sent to [filter2idl]{.title-ref}.</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>FilterResolved::And(l) =&gt; { </span><span> // First, setup the two filter lists. We always apply AndNot after positive </span><span> // and terms. </span><span> let (f_andnot, f_rem): (Vec&lt;_&gt;, Vec&lt;_&gt;) = l.iter().partition(|f| f.is_andnot()); </span><span> </span><span> // We make this an iter, so everything comes off in order. if we used pop it means we </span><span> // pull from the tail, which is the WORST item to start with! </span><span> let mut f_rem_iter = f_rem.iter(); </span><span> </span><span> // Setup the initial result. </span><span> let (mut cand_idl, fp) = match f_rem_iter.next() { </span><span> Some(f) =&gt; self.filter2idl(au, f, thres)?, </span><span> None =&gt; { </span><span> lfilter_error!(au, &quot;WARNING: And filter was empty, or contains only AndNot, can not evaluate.&quot;); </span><span> return Ok((IDL::Indexed(IDLBitRange::new()), FilterPlan::Invalid)); </span><span> } </span><span> }; </span><span> ... </span></code></pre> <p>The first term we encounter is [{&quot;eq&quot;: [&quot;class&quot;, &quot;account&quot;, true]}]{.title-ref}. At this point [filter2idl]{.title-ref} is able to <a href="https://github.com/kanidm/kanidm/blob/dbfe87e675beac7fd931a445fd80cf439c2c6e61/kanidmd/src/lib/be/mod.rs#L123">request the id list</a> from the lower levels.</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>FilterResolved::Eq(attr, value, idx) =&gt; { </span><span> if *idx { </span><span> // Get the idx_key </span><span> let idx_key = value.get_idx_eq_key(); </span><span> // Get the idl for this </span><span> match self </span><span> .get_idlayer() </span><span> .get_idl(au, attr, &amp;IndexType::EQUALITY, &amp;idx_key)? </span><span> { </span><span> Some(idl) =&gt; ( </span><span> IDL::Indexed(idl), </span><span> FilterPlan::EqIndexed(attr.to_string(), idx_key), </span><span> ), </span><span> None =&gt; (IDL::ALLIDS, FilterPlan::EqCorrupt(attr.to_string())), </span><span> } </span><span> } else { </span><span> // Schema believes this is not indexed </span><span> (IDL::ALLIDS, FilterPlan::EqUnindexed(attr.to_string())) </span><span> } </span><span>} </span></code></pre> <p>The first level that is able to serve the request for the key to be resolved is the <a href="https://github.com/kanidm/kanidm/blob/dbfe87e675beac7fd931a445fd80cf439c2c6e61/kanidmd/src/lib/be/idl_arc_sqlite.rs#L178">ARCache layer</a>. This tries to lookup the combination of (&quot;class&quot;, &quot;account&quot;, eq) in the cache. If found it is returned to the caller. If not, it is requested from the <a href="https://github.com/kanidm/kanidm/blob/master/kanidmd/src/lib/be/idl_sqlite.rs#L220">sqlite layer</a>.</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>let cache_key = IdlCacheKey { </span><span> a: $attr.to_string(), </span><span> i: $itype.clone(), </span><span> k: $idx_key.to_string(), </span><span>}; </span><span>let cache_r = $self.idl_cache.get(&amp;cache_key); </span><span>// If hit, continue. </span><span>if let Some(ref data) = cache_r { </span><span> ltrace!( </span><span> $audit, </span><span> &quot;Got cached idl for index {:?} {:?} -&gt; {}&quot;, </span><span> $itype, </span><span> $attr, </span><span> data </span><span> ); </span><span> return Ok(Some(data.as_ref().clone())); </span><span>} </span><span>// If miss, get from db *and* insert to the cache. </span><span>let db_r = $self.db.get_idl($audit, $attr, $itype, $idx_key)?; </span><span>if let Some(ref idl) = db_r { </span><span> $self.idl_cache.insert(cache_key, Box::new(idl.clone())) </span><span>} </span></code></pre> <p>This sqlite layer performs the select from the &quot;idx_&lt;type&gt;_&lt;attr&gt;&quot; table, and then deserialises the stored id list (IDL).</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>let mut stmt = self.get_conn().prepare(query.as_str()).map_err(|e| { </span><span> ladmin_error!(audit, &quot;SQLite Error {:?}&quot;, e); </span><span> OperationError::SQLiteError </span><span>})?; </span><span>let idl_raw: Option&lt;Vec&lt;u8&gt;&gt; = stmt </span><span> .query_row_named(&amp;[(&quot;:idx_key&quot;, &amp;idx_key)], |row| row.get(0)) </span><span> // We don&#39;t mind if it doesn&#39;t exist </span><span> .optional() </span><span> .map_err(|e| { </span><span> ladmin_error!(audit, &quot;SQLite Error {:?}&quot;, e); </span><span> OperationError::SQLiteError </span><span> })?; </span><span> </span><span>let idl = match idl_raw { </span><span> Some(d) =&gt; serde_cbor::from_slice(d.as_slice()) </span><span> .map_err(|_| OperationError::SerdeCborError)?, </span><span> // We don&#39;t have this value, it must be empty (or we </span><span> // have a corrupted index ..... </span><span> None =&gt; IDLBitRange::new(), </span><span>}; </span></code></pre> <p>The IDL is returned and cached, then returned to the [filter2idl]{.title-ref} caller. At this point the IDL is the &quot;partial candidate set&quot;. It contains the ID numbers of entries that we know partially match this query at this point. Since the first term is [{&quot;eq&quot;: [&quot;class&quot;, &quot;account&quot;, true]}]{.title-ref} the current candidate set is [[1, 2, ...]]{.title-ref}.</p> <p>The <a href="https://github.com/kanidm/kanidm/blob/dbfe87e675beac7fd931a445fd80cf439c2c6e61/kanidmd/src/lib/be/mod.rs#L284">and</a> path in [filter2idl]{.title-ref} continues, and the next term encountered is [{&quot;eq&quot;: [&quot;name&quot;: &quot;william&quot;, true]}]{.title-ref}. This resolves into another IDL. The two IDL's are merged through an [and]{.title-ref} operation leaving only the ID numbers that were present in both.</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>(IDL::Indexed(ia), IDL::Indexed(ib)) =&gt; { </span><span> let r = ia &amp; ib; </span><span> ... </span></code></pre> <p>For this example this means in our example that the state of r(esult set) is the below;</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>res = ia &amp; ib; </span><span>res = [1, 2, ....] &amp; [1, ]; </span><span>res == [1, ] </span></code></pre> <p>We know that only the entry with [ID == 1]{.title-ref} matches both [name = william]{.title-ref} and [class = account]{.title-ref}.</p> <p>We now perform a check called the &quot;filter threshold check&quot;. If the number of ID's in the IDL is less than a certain number, we can <em>shortcut</em> and return early even though we are not finished processing.</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>if r.len() &lt; thres &amp;&amp; f_rem_count &gt; 0 { </span><span> // When below thres, we have to return partials to trigger the entry_no_match_filter check. </span><span> let setplan = FilterPlan::AndPartialThreshold(plan); </span><span> return Ok((IDL::PartialThreshold(r), setplan)); </span><span>} else if r.is_empty() { </span><span> // Regardless of the input state, if it&#39;s empty, this can never </span><span> // be satisfied, so return we are indexed and complete. </span><span> let setplan = FilterPlan::AndEmptyCand(plan); </span><span> return Ok((IDL::Indexed(IDLBitRange::new()), setplan)); </span><span>} else { </span><span> IDL::Indexed(r) </span><span>} </span></code></pre> <p>This is because the IDL is now small, and continuing to load more indexes may cost more time and resources. The IDL can only ever shrink or stay the same from this point, never expand, so we know it must stay small.</p> <p>However, you may correctly have deduced that there are still two terms we must check. That is the terms contained within the [andnot]{.title-ref} of the query. I promise you, we will check them :)</p> <p>So at this point we now step out of [filter2idl]{.title-ref} and begin the process of post-processing the results we have.</p> <h2 id="resolving-the-partial-set">Resolving the Partial Set</h2> <p>We check the way that the <a href="https://github.com/kanidm/kanidm/blob/dbfe87e675beac7fd931a445fd80cf439c2c6e61/kanidmd/src/lib/be/mod.rs#L498">IDL is tagged</a> so that we understand what post processing is required, and check some security controls. If the search was unindexed aka [ALLIDS]{.title-ref}, and if the account is not allowed to access fully unindexed searches, then we return a failure at this point. We also now check if the query was [Partial(ly)]{.title-ref} unindexed, and if it is, we assert limits over the number of entries we may load and test.</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>match &amp;idl { </span><span> IDL::ALLIDS =&gt; { </span><span> if !erl.unindexed_allow { </span><span> ladmin_error!(au, &quot;filter (search) is fully unindexed, and not allowed by resource limits&quot;); </span><span> return Err(OperationError::ResourceLimit); </span><span> } </span><span> } </span><span> IDL::Partial(idl_br) =&gt; { </span><span> if idl_br.len() &gt; erl.search_max_filter_test { </span><span> ladmin_error!(au, &quot;filter (search) is partial indexed and greater than search_max_filter_test allowed by resource limits&quot;); </span><span> return Err(OperationError::ResourceLimit); </span><span> } </span><span> } </span><span> IDL::PartialThreshold(_) =&gt; { </span><span> // Since we opted for this, this is not the fault </span><span> // of the user and we should not penalise them by limiting on partial. </span><span> } </span><span> IDL::Indexed(idl_br) =&gt; { </span><span> // We know this is resolved here, so we can attempt the limit </span><span> // check. This has to fold the whole index, but you know, class=pres is </span><span> // indexed ... </span><span> if idl_br.len() &gt; erl.search_max_results { </span><span> ladmin_error!(au, &quot;filter (search) is indexed and greater than search_max_results allowed by resource limits&quot;); </span><span> return Err(OperationError::ResourceLimit); </span><span> } </span><span> } </span><span>}; </span></code></pre> <p>We then load the related entries from the IDL we have. Initially, this is called through the entry cache of the ARCache.</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>let entries = self.get_idlayer().get_identry(au, &amp;idl).map_err(|e| { </span><span> ladmin_error!(au, &quot;get_identry failed {:?}&quot;, e); </span><span> e </span><span>})?; </span></code></pre> <p>As many entries as possible are <a href="https://github.com/kanidm/kanidm/blob/dbfe87e675beac7fd931a445fd80cf439c2c6e61/kanidmd/src/lib/be/idl_arc_sqlite.rs#L93">loaded from the ARCache</a>. The remaining ID's that were missed are stored in a secondary IDL set. The missed entry set is then submitted to <a href="https://github.com/kanidm/kanidm/blob/master/kanidmd/src/lib/be/idl_sqlite.rs#L99">the sqlite layer</a> where the entries are loaded and deserialised. An important part of the ARCache is to keep fully inflated entries in memory, to speed up the process of retrieving these. Real world use shows this can have orders of magnitude of impact on performance by just avoiding this deserialisation step, but also that we avoid IO to disk.</p> <p>The entry set is now able to be checked. If the IDL was [Indexed]{.title-ref} no extra work is required, and we can just return the values. But in all other cases we must apply the filter test. The filter test is where the terms of the filter are checked against each entry to determine if they match and are part of the set.</p> <p>This is where the partial threshold is important - that the act of processing the remaining indexes may be more expensive than applying the filter assertions to the subset of entries in memory. It's also why filter optimisation matters. If a query can be below the threshold sooner, than we can apply the filter test earlier and we reduce the number of indexes we must load and keep cached. This helps performance and cache behaviour.</p> <p>The <a href="https://github.com/kanidm/kanidm/blob/dbfe87e675beac7fd931a445fd80cf439c2c6e61/kanidmd/src/lib/entry.rs#L1663">filter test</a> applies the terms of the filter to the entry, using the same rules as the indexing process to ensure consistent results. This gives us a true/false result, which lets us know if the entry really does match and should become part of the final candidate set.</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>fn search(...) { </span><span> ... </span><span> IDL::Partial(_) =&gt; lperf_segment!(au, &quot;be::search&lt;entry::ftest::partial&gt;&quot;, || { </span><span> entries </span><span> .into_iter() </span><span> .filter(|e| e.entry_match_no_index(&amp;filt)) </span><span> .collect() </span><span> }), </span><span> ... </span><span>} </span><span> </span><span>fn entry_match_no_index_inner(&amp;self, filter: &amp;FilterResolved) -&gt; bool { </span><span> match filter { </span><span> FilterResolved::Eq(attr, value, _) =&gt; self.attribute_equality(attr.as_str(), value), </span><span> FilterResolved::Sub(attr, subvalue, _) =&gt; { </span><span> self.attribute_substring(attr.as_str(), subvalue) </span><span> } </span><span> ... </span><span> } </span><span>} </span></code></pre> <p>It is now at this point that we finally have the fully resolved set of entries, in memory as a result set from the backend. These are returned to the query server's [search]{.title-ref} function.</p> <h2 id="access-controls">Access Controls</h2> <p>Now the process of <a href="https://github.com/kanidm/kanidm/blob/dbfe87e675beac7fd931a445fd80cf439c2c6e61/kanidmd/src/lib/server.rs#L168">applying</a> access controls begins. There are two layers of access controls as applied in kanidm. The first is <em>which entries are you allowed to see</em>. The second is <em>within an entry, what attributes may you see</em>. There is a reason for this seperation. The seperation exists so that when an internal search is performed on behalf of the user, we retrieve the set of entries you can see, but the server internally then performs the operation on your behalf and itself has access to see all attributes. If you wish to see this in action, it's a critical part of how <a href="https://github.com/kanidm/kanidm/blob/dbfe87e675beac7fd931a445fd80cf439c2c6e61/kanidmd/src/lib/server.rs#L1290">modify</a> and <a href="https://github.com/kanidm/kanidm/blob/dbfe87e675beac7fd931a445fd80cf439c2c6e61/kanidmd/src/lib/server.rs#L988">delete</a> both function, where you can only change or delete within your visible entry scope.</p> <p>The first stage is <a href="https://github.com/kanidm/kanidm/blob/master/kanidmd/src/lib/access.rs#L376">search_filter_entries</a>. This is the function that checks what entries you <em>may</em> see. This checks that you have the rights to see specific attributes (ie can you see name?), which then affects, &quot;could you possibly have queried for this?&quot;.</p> <p>Imagine for example, that we search for &quot;password = X&quot; (which kanidm disallows but anyway ...). Even if you could not read password, the act of testing the equality, if an entry was returned you would know now about the value or association to a user since the equality condition held true. This is a security risk for information disclosure.</p> <p>The first stage of access controls is <a href="https://github.com/kanidm/kanidm/blob/master/kanidmd/src/lib/access.rs#L397">what rules apply to your authenticated user</a>. There may be thousands of access controls in the system, but only some may related to your account.</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>let related_acp: Vec&lt;&amp;AccessControlSearch&gt; = </span><span> lperf_segment!(audit, &quot;access::search_filter_entries&lt;related_acp&gt;&quot;, || { </span><span> search_state </span><span> .iter() </span><span> .filter(|acs| { </span><span> let f_val = acs.acp.receiver.clone(); </span><span> match f_val.resolve(&amp;se.event, None) { </span><span> Ok(f_res) =&gt; rec_entry.entry_match_no_index(&amp;f_res), </span><span> Err(e) =&gt; { </span><span> ... </span><span> } </span><span> } </span><span> }) </span><span> .collect() </span><span> }); </span></code></pre> <p>The next stage is to determine <a href="https://github.com/kanidm/kanidm/blob/master/kanidmd/src/lib/access.rs#L440">what attributes did you request to filter on</a>. This is why [filter_orig]{.title-ref} is stored in the event. We must test against the filter as intended by the caller, not the filter as executed. This is because the filter as executed may have been transformed by the server, using terms the user does not have access to.</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>let requested_attrs: BTreeSet&lt;&amp;str&gt; = se.filter_orig.get_attr_set(); </span></code></pre> <p>Then for each entry, the set of allowed attributes is determined. If the user related access control also holds rights oven the entry in the result set, the set of attributes it grants read access over is appended to the &quot;allowed&quot; set. This repeats until the set of related access controls is exhausted.</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>let allowed_entries: Vec&lt;Entry&lt;EntrySealed, EntryCommitted&gt;&gt; = </span><span> entries </span><span> .into_iter() </span><span> .filter(|e| { </span><span> </span><span> let allowed_attrs: BTreeSet&lt;&amp;str&gt; = related_acp.iter() </span><span> .filter_map(|acs| { </span><span> ... </span><span> if e.entry_match_no_index(&amp;f_res) { </span><span> // add search_attrs to allowed. </span><span> Some(acs.attrs.iter().map(|s| s.as_str())) </span><span> } else { </span><span> None </span><span> } </span><span> ... </span><span> }) </span><span> .collect(); </span><span> </span><span> let decision = requested_attrs.is_subset(&amp;allowed_attrs); </span><span> lsecurity_access!(audit, &quot;search attr decision --&gt; {:?}&quot;, decision); </span><span> decision </span><span> }) </span></code></pre> <p>This now has created a set of &quot;attributes this person can see&quot; on this entry based on all related rules. The requested attributes are compared to the attributes you may see, and if requested is a subset or equal, then the entry is allowed to be returned to the user.</p> <p>If there is even a single attribute in the query you do not have the rights to see, then the entry is disallowed from the result set. This is because if you can not see that attribute, you must not be able to apply a filter test to it.</p> <p>To give a worked example, consider the entry from before. We also have three access controls:</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>applies to: all users </span><span>over: pres class </span><span>read attr: class </span><span> </span><span>applies to: memberof admins </span><span>over: entries where class = account </span><span>read attr: name, displayname </span><span> </span><span>applies to: memberof radius_servers </span><span>over: entries where class = account </span><span>read attr: radius secret </span></code></pre> <p>Our current authenticated user (let's assume it's also &quot;name=william&quot;), would only have the first two rules apply. As we search through the candidate entries, the &quot;all users&quot; rule would match our entry, which means class is added to the allowed set. Then since william is memberof admins, they also have read to name, and displayname. Since the target entry is class=account then name and displayname are also added to the allowed set. But since william is <em>not</em> a member of radius_servers, we don't get to read radius secrets.</p> <p>At this point the entry set is reduced to the set of entries the user was <em>able</em> to have applied filter tests too, and is returned.</p> <p>The query server then unwinds to [search_ext]{.title-ref} where the second stage of access controls is now checked. This calls <a href="https://github.com/kanidm/kanidm/blob/master/kanidmd/src/lib/access.rs#L524">search_filter_entry_attributes</a> which is responsible for changing an entry in memory to remove content that the user may not see. A key difference is this line:</p> <p>Again, the set of related access controls is generated, and then applied to each entry to determine if they are in scope. This builds a set of &quot;attributes the user can see, per entry&quot;. This is then applied to the entry to reduction function, which removes any attribute <em>not</em> in the allowed set.</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>e.reduce_attributes(&amp;allowed_attrs) </span></code></pre> <p>A clear example is when you attempt to view yourself vs when you view another persons account as there are permissions over self that exist, which do not apply to others. You may view your own legalname field, but not the legalname of another person.</p> <p>The entry set is finally returned and turned into a JSON entry for transmission to the client. Hooray!</p> <h2 id="conclusion">Conclusion</h2> <p>There is a lot that goes into a query being processed in a database. But like all things in computing since it was created by a person, any other person must be able to understand it. It's always amazing that this whole process can be achieved in fractions of a second, in parallel, and so reliably.</p> <p>There is so much more involved in this process too. The way that a write operation is performed to extract correct index values, the way that the database reloads the access control cache based on changes, and even how the schema is loaded and constructed. Ontop of all this, a complete identity management stack is built that can allow authentication through wireless, machines, ssh keys and more.</p> <p>If you are interested more in databases and <a href="https://github.com/kanidm/kanidm">Kanidm</a> please get in contact!</p> Using SUSE Leap Enterprise with Docker Wed, 26 Aug 2020 00:00:00 +0000 Unknown https://fy.blackhats.net.au/blog/2020-08-26-using-suse-leap-enterprise-with-docker/ https://fy.blackhats.net.au/blog/2020-08-26-using-suse-leap-enterprise-with-docker/ <h1 id="using-suse-leap-enterprise-with-docker">Using SUSE Leap Enterprise with Docker</h1> <p>It's a little bit annoying to connect up all the parts for this. If you have a SLE15 system then credentials for SCC are automatically passed into containers via secrets.</p> <p>But if you are on a non-SLE base, like myself with MacOS or OpenSUSE you'll need to provide these to the container in another way. The documentation is a bit tricky to search and connect up what you need but in summary:</p> <ul> <li>Get [/etc/SUSEConnect]{.title-ref} and [/etc/zypp/credentials.d/SCCcredentials]{.title-ref} from an SLE install that has been registered. The SLE version does not matter.</li> <li>Mount them into the image:</li> </ul> <pre data-lang="bash" style="background-color:#2b303b;color:#c0c5ce;" class="language-bash "><code class="language-bash" data-lang="bash"><span> </span><span style="color:#bf616a;">docker</span><span> ...</span><span style="color:#bf616a;"> -v</span><span> /scc/SUSEConnect:/etc/SUSEConnect \ </span><span style="color:#bf616a;"> -v</span><span> /scc/SCCcredentials:/etc/zypp/credentials.d/SCCcredentials \ </span><span> ... </span><span> </span><span style="color:#bf616a;">registry.suse.com/suse/sle15:15.2 </span></code></pre> <p>Now you can use the images from <a href="https://registry.suse.com/">the SUSE registry</a>. For example [docker pull registry.suse.com/suse/sle15:15.2]{.title-ref} and have working zypper within them.</p> <p>If you want to add extra modules to your container (you can list what's available with container-suseconnect from an existing SLE container of the same version), you can do this by adding environment variables at startup. For example, to add dev tools like gdb:</p> <pre data-lang="bash" style="background-color:#2b303b;color:#c0c5ce;" class="language-bash "><code class="language-bash" data-lang="bash"><span> </span><span style="color:#bf616a;">docker</span><span> ...</span><span style="color:#bf616a;"> -e</span><span> ADDITIONAL_MODULES=sle-module-development-tools \ </span><span style="color:#bf616a;"> -v</span><span> /scc/SUSEConnect:/etc/SUSEConnect \ </span><span style="color:#bf616a;"> -v</span><span> /scc/SCCcredentials:/etc/zypp/credentials.d/SCCcredentials \ </span><span> ... </span><span> </span><span style="color:#bf616a;">registry.suse.com/suse/sle15:15.2 </span></code></pre> <p>This also works during builds to add extra modules.</p> <p>HINT: SUSEConnect and SCCcredentials and not version dependent so will work in any image version.</p> Windows Hello in Webauthn-rs Mon, 24 Aug 2020 00:00:00 +0000 Unknown https://fy.blackhats.net.au/blog/2020-08-24-windows-hello-in-webauthn-rs/ https://fy.blackhats.net.au/blog/2020-08-24-windows-hello-in-webauthn-rs/ <h1 id="windows-hello-in-webauthn-rs">Windows Hello in Webauthn-rs</h1> <p>Recently I've been working again on <a href="https://crates.io/crates/webauthn-rs">webauthn-rs</a>, as a member of the community wants to start using it in production for a service. So far the development of the library has been limited to the test devices that I own, but now this pushes me toward implementing true fido compliance.</p> <p>A really major part of this though was that a lot of their consumers use windows, which means support windows hello.</p> <h2 id="a-background-on-webauthn">A background on webauthn</h2> <p>Webauthn itself is not a specification for the cryptographic operations required for authentication using an authenticator device, but a specification that wraps other techniques to allow a variety of authenticators to be used exposing their &quot;native&quot; features.</p> <p>The authentication side of webauthn is reasonably simple in this way. The server stores a public key credential associated to a user. During authentication the server provides a challenge which the authenticator signs using it's private key. The server can then verify using it's copy of the challenge, and the public key that the authentication must have come from that credentials. Of course like anything there is a little bit of magic in here around how the authenticators store credentials that allows other properties to be asserted, but that's beyond the scope of this post.</p> <p>The majority of the webauthn specification is around the process of registering credentials and requesting specific properties to exist in the credentials. Some of these properties are optional hints (resident keys, authenticator attachment) and some of these properties are enforced (user verification so that the credential is a true MFA). Beyond these there is also a process for the authenticator to provide information about it's source and trust. This process is attestation and has multiple different formats and details associated.</p> <p>It's interesting to note that for most deployments of webauthn, attestation is not required by the attestation conveyance preference, and generally provides little value to these deployments. For many sites you only need to know that a webauthn authenticator is in use. However attestation allows environments with strict security requirements to verify and attest the legitimacy of, and make and model of authenticators in use. (An interesting part of webauthn is how much of it seems to be Google and Microsoft internal requirements leaking into a specification, just saying).</p> <p>This leads to what is effectively, most of the code in webauthn-rs - attestation.rs.</p> <h2 id="windows-hello">Windows Hello</h2> <p>Windows Hello is Microsoft's equivalent to TouchID on iOS. Using a Trusted Platform Module (TPM) as a tamper-resistant secure element, it allows devices such as a Windows Surface to perform cryptographic operations. As Microsoft is attempting to move to a passwordless future (which honestly, I'm on board for and want to support in Kanidm), this means they want to support Webauthn on as many of their devices as possible. Microsoft even defines in their hardware requirements for Windows 10 Home, Pro, Education and Enterprise that <a href="https://docs.microsoft.com/en-us/windows-hardware/design/minimum/minimum-hardware-requirements-overview">as of July 28, 2016, all new device models, lines or series ... a component which implements the TPM 2.0 must be present and enabled by default from this effective date.</a>. This is pretty major as this means that slowly MS have been ensuring that <em>all</em> consumer and enterprise devices are steadily moving to a point where passwordless is a viable future. Microsoft state that they use TPMv2 for many reasons, but a defining one is: <a href="https://docs.microsoft.com/en-us/windows/security/information-protection/tpm/tpm-recommendations">The TPM 1.2 spec only allows for the use of RSA and the SHA-1 hashing algorithm</a> which is now considered broken.</p> <p>Of course, if you have noticed this means that TPM's are involved. Webauthn supports a TPM attestation path, and that means I have to implement it.</p> <h2 id="once-more-into-the-abyss">Once more into the abyss</h2> <p>Reading the <a href="https://www.w3.org/TR/webauthn/#tpm-attestation">Webauthn spec for TPM attestation</a> it pointed me to the TPMv2.0 specification part1, part2 and part3. I will spare you from this as there is a sum total of 861 pages between these documents, and the Webauthn spec while it only references a few parts, manages to then create a set of expanding references within these documents. To make it even more enjoyable, text search is mostly broken in these documents, meaning that trying to determine the field contents and types involves a lot of manual-eyeball work.</p> <p>TPM's structures are packed C structs which means that they can be very tricky to parse. They use u16 identifiers to switch on unions, and other fun tricks that we love to see from C programs. These u16's often have some defined constants which are valid choices, such as TPM_ALG_ID, which allows switching on which cryptographic algorithms are in use. Some stand out parts of this section were as follows.</p> <p>Unabashed optimism:</p> <p><code>TPM_ALG_ERROR 0x0000 // Should not occur</code></p> <p>Broken Crypto</p> <p><code>TPM_ALG_SHA1 0x0004 // The SHA1 Algorithm</code></p> <p>Being the boomer equivalent of <a href="https://auth0.com/blog/critical-vulnerabilities-in-json-web-token-libraries/">JWT</a></p> <p><code>TPM_ALG_NULL 0x0010 // The NULL Algorithm</code></p> <p>And supporting the latest in modern cipher suites</p> <p><code>TPM_ALG_XOR 0x000A // TCG TPM 2.0 library specification - the XOR encryption algorithm.</code></p> <p><code>ThE XOR eNcRyPtIoN aLgoRitHm.</code></p> <p>Some of the structures are even quite fun to implement, such as TPMT_SIGNATURE, where a matrix of how to switch on it is present where the first two bytes when interpreted as a u16, define a TPM_ALG_ID where, if it the two bytes are not in a set of the TPM_ALG_ID then the whole blob including leading two bytes is actually just a blob of hash. It would certainly be unfortunate if in the interest of saving two bytes that my hash accidentally emited data where the first two bytes were accidentally a TPM_ALG_ID causing a parser to overflow.</p> <p>I think the cherry on all of this though, is that despite Microsoft requiring TPMv2.0 to move away from RSA and SHA-1, that when I checked the attestation signatures for a Windows Hello device I had to implement the following:</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>COSEContentType::INSECURE_RS1 =&gt; { </span><span> hash::hash(hash::MessageDigest::sha1(), input) </span><span> .map(|dbytes| Vec::from(dbytes.as_ref())) </span><span> .map_err(|e| WebauthnError::OpenSSLError(e)) </span><span>} </span></code></pre> <h2 id="conclusion">Conclusion</h2> <p>Saying this, I'm happy that Windows Hello is now in Webauthn-rs. The actual Webauthn authentication flows DO use secure algorithms (RSA2048 + SHA256 and better), it is only in the attestation path that some components are signed by SHA1. So please use <a href="https://crates.io/crates/webauthn-rs">webauthn-rs</a>, and do use Windows Hello with it!</p> User gesture is not detected - using iOS TouchID with webauthn-rs Wed, 12 Aug 2020 00:00:00 +0000 Unknown https://fy.blackhats.net.au/blog/2020-08-12-user-gesture-is-not-detected-using-ios-touchid-with-webauthn-rs/ https://fy.blackhats.net.au/blog/2020-08-12-user-gesture-is-not-detected-using-ios-touchid-with-webauthn-rs/ <h1 id="user-gesture-is-not-detected-using-ios-touchid-with-webauthn-rs">User gesture is not detected - using iOS TouchID with webauthn-rs</h1> <p>I was recently contacted by a future user of <a href="https://github.com/kanidm/webauthn-rs">webauthn-rs</a> who indicated that the library may not currently support Windows Hello as an authenticator. This is due to the nature of the device being a platform attached authenticator and that webauthn-rs at the time did not support attachment preferences.</p> <p>As I have an ipad, and it's not a primary computing device I decided to upgrade to iPadOS 14 beta to try out webauthn via touch (and handwriting support).</p> <h2 id="the-issue">The Issue</h2> <p>After watching <a href="https://developer.apple.com/videos/play/wwdc2020/10670/">Jiewen's WWDC presentation</a> about using TouchID with webauthn, I had a better idea about some of the server side requirements to work with this.</p> <p>Once I started to test though, I would recieve the following error in the safari debug console.</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>User gesture is not detected. To use the platform authenticator, </span><span>call &#39;navigator.credentials.create&#39; within user activated events. </span></code></pre> <p>I was quite confused by this error - a user activated event seems to be a bit of an unknown term, and asking other people they also didn't quite know what it meant. My demo site was using a button input with onclick event handlers to call javascript similar to the following:</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>function register() { </span><span>fetch(REG_CHALLENGE_URL + username, {method: &quot;POST&quot;}) </span><span> .then(res =&gt; { </span><span> ... // error handling </span><span> }) </span><span> .then(res =&gt; res.json()) </span><span> .then(challenge =&gt; { </span><span> challenge.publicKey.challenge = fromBase64(challenge.publicKey.challenge); </span><span> challenge.publicKey.user.id = fromBase64(challenge.publicKey.user.id); </span><span> return navigator.credentials.create(challenge) </span><span> .then(newCredential =&gt; { </span><span> console.log(&quot;PublicKeyCredential Created&quot;); </span><span> .... </span><span> return fetch(REGISTER_URL + username, { </span><span> method: &quot;POST&quot;, </span><span> body: JSON.stringify(cc), </span><span> headers: { </span><span> &quot;Content-Type&quot;: &quot;application/json&quot;, </span><span> }, </span><span> }) </span><span> }) </span></code></pre> <p>This works happily in Firefox and Chrome, and for iPadOS it event works with my yubikey 5ci.</p> <p>I investigated further to determine if the issue was in the way I was presenting the registration to the [navigator.credentials.create]{.title-ref} function. Comparing to webauthn.io (which does work with TouchID on iPadOS 14 beta), I noticed some subtle differences but nothing that should cause an issue like this.</p> <p>After much pacing, thinking and asking for help I eventually gave in and went to the source of webkit</p> <h2 id="the-solution">The Solution</h2> <p>Reading through the webkit source I noted that the check within the code was looking for association of how the event was initiated. This comes from a context that is available within the browser. This got me to think about the fact that the fetch api is <em>async</em>, and I realised at this point that webauthn.io was using the [jQuery.ajax]{.title-ref} apis. I altered my demo to use the same, and it began to work with TouchID. That meant that the user activation was being lost over the async boundary to the fetch API. (note: it's quite reasonable to expect user interaction to use [navigator.credentials]{.title-ref} to prevent tricking or manipulating users into activating their webauthn devices).</p> <p>I emailed Jiewen, who responded overnight and informed me that this is an issue, and it's being tracked in the <a href="https://bugs.webkit.org/show_bug.cgi?id=214722">webkit bugzilla</a> . He assures me that it will be resolved in a future release. Many thanks to him for helping me with this issue!</p> <p>At this point I now know that TouchID will work with webauthn-rs, and I can submit some updates to the library to help support this.</p> <h2 id="notes-on-webauthn-with-touchid">Notes on Webauthn with TouchID</h2> <p>It's worth pointing out a few notes from the WWDC talk, and the differences I have observed with webauthn on real devices.</p> <p>In the presentation it is recommended that in your Credential Creation Options, that you (must?) define the options listed to work with TouchID</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>authenticatorSelection: { authenticatorAttachment: &quot;platform&quot; }, </span><span>attestation: &quot;direct&quot; </span></code></pre> <p>It's worth pointing out that authenticatorAttachment is only a <em>hint</em> to the client to which credentials it should use. This allows your web page to streamline the UI flow (such as detection of platform key and then using that to toggle the authenticatorAttachment), but it's not an enforced security policy. <em>There is no part of the attestation response that indicates the attachement method</em>. The only way to determine that the authenticator is a platform authenticator would be in attestation &quot;direct&quot; to validate the issuing CA or the device's AAGUID match the expectations you hold for what authenticators can be used within your organisation or site.</p> <p>Additionally, TouchID does work with <em>no</em> authenticatorAttachment hint (safari prompts if you want to use an external key or TouchID), and that [attestation: &quot;none&quot;]{.title-ref} also works. This means that a minimised and default set of Credential Creation Options will allow you to work with the majority of webauthn devices.</p> <p>Finally, the WWDC video glosses over the server side process. Be sure to follow the w3c standard for verifying attestations, or use a library that implementes this standard (such as <a href="https://github.com/kanidm/webauthn-rs">webauthn-rs</a> or <a href="https://github.com/duo-labs/webauthn/">duo-labs go webauthn</a>). I'm sure that other libraries exist, but it's critical that they follow the w3c process as webauthn is quite complex and fiddly to implement in a correct and secure manner.</p> docker buildx for multiarch builds Thu, 06 Aug 2020 00:00:00 +0000 Unknown https://fy.blackhats.net.au/blog/2020-08-06-docker-buildx-for-multiarch-builds/ https://fy.blackhats.net.au/blog/2020-08-06-docker-buildx-for-multiarch-builds/ <h1 id="docker-buildx-for-multiarch-builds">docker buildx for multiarch builds</h1> <p>I have been previously building Kanidm with plain docker build, but recently a community member wanted to be able to run kanidm on arm64. That meant that I needed to go down the rabbit hole of how to make this work ...</p> <h2 id="what-not-to-do">What not to do ...</h2> <p>There is a previous method of using manifest files to allow multiarch uploads. It's pretty messy but it works, so this is an option if you want to investigate but I didn't want to pursue it.</p> <p>Bulidx exists and I got it working on my linux machine with the steps from <a href="https://www.docker.com/blog/getting-started-with-docker-for-arm-on-linux/">here</a> but the build took more than 3 hours, so I don't recommend it if you plan to do anything intense or frequently.</p> <h2 id="buildx-cluster">Buildx cluster</h2> <p>Docker has a cross-platform building toolkit called buildx which is currently tucked into the experimental features. It can be enabled on docker for mac in the settings (note: you only need experimental support on the coordinating machine aka your workstation).</p> <p>Rather than follow the <a href="https://docs.docker.com/buildx/working-with-buildx/">official docs</a> this will branch out. The reason is that buildx in the official docs uses qemu-aarch64 translation which is very slow and energy hungry, taking a long time to produce builds. As mentioned already I was seeing in excess of 3 hours for aarch64 on my builder VM or my mac.</p> <p>Instead, in this configuration I will use my mac as a coordinator, and an x86_64 VM and a rock64pro as builder nodes, so that the builds are performed on native architecture machines.</p> <p>First we need to configure our nodes. In [/etc/docker/daemon.json]{.title-ref} we need to expose our docker socket to our mac. I have done this with the following:</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>{ </span><span> &quot;hosts&quot;: [&quot;unix:///var/run/docker.sock&quot;, &quot;tcp://0.0.0.0:2376&quot;] </span><span>} </span></code></pre> <p><em>WARNING</em>: This configuration is HIGHLY INSECURE. This exposes your docker socket to the network with no authentication, which is equivalent to un-authenticated root access. I have done this because my builder nodes are on an isolated and authenticated VLAN of my home network. You should either do similar or use TLS authentication.</p> <p>NOTE: The [ssh://]{.title-ref} transport does not work for docker buildx. No idea why but it don't.</p> <p>Once this is done restart docker on the two builder nodes.</p> <p>Now we can configure our coordinator machine. We need to check buildx is present:</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>docker buildx --help </span></code></pre> <p>We then want to create a new builder instance and join our nodes to it. We can use the DOCKER_HOST environment variable for this:</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>DOCKER_HOST=tcp://x.x.x.x:2376 docker buildx create --name cluster </span><span>DOCKER_HOST=tcp://x.x.x.x:2376 docker buildx create --name cluster --append </span></code></pre> <p>We can then startup and bootstrap the required components with:</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>docker buildx use cluster </span><span>docker buildx inspect --bootstrap </span></code></pre> <p>We should see output like:</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>Name: cluster </span><span>Driver: docker-container </span><span> </span><span>Nodes: </span><span>Name: cluster0 </span><span>Endpoint: tcp://... </span><span>Status: running </span><span>Platforms: linux/amd64, linux/386 </span><span> </span><span>Name: cluster1 </span><span>Endpoint: tcp://... </span><span>Status: running </span><span>Platforms: linux/arm64, linux/arm/v7, linux/arm/v6 </span></code></pre> <p>If we are happy with this we can make this the default builder.</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>docker buildx use cluster --default </span></code></pre> <p>And you can now use it to build your images such as:</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>docker buildx build --push --platform linux/amd64,linux/arm64 -f Dockerfile -t &lt;tag&gt; . </span></code></pre> <p>Now I can build my multiarch images much quicker and efficently!</p> Developer Perspective on Docker Mon, 13 Jul 2020 00:00:00 +0000 Unknown https://fy.blackhats.net.au/blog/2020-07-13-developer-perspective-on-docker/ https://fy.blackhats.net.au/blog/2020-07-13-developer-perspective-on-docker/ <h1 id="developer-perspective-on-docker">Developer Perspective on Docker</h1> <p>A good mate of mine <a href="https://ronamosa.io/">Ron Amosa</a> put a question up on twitter about what do developers think Docker brings to the table. I'm really keen to see what he has to say (his knowledge of CI/CD and Kubernetes is amazing by the way!), but I thought I'd answer his question from my view as a software engineer.</p> <p><em>Docker provides resource isolation and management to applications</em></p> <p>Lets break that down.</p> <h2 id="what-is-a-resource-what-is-an-application">What is a resource? What is an application?</h2> <p>It doesn't matter what kind of application we write: A Rust command line tool, an embedded database in C, or a webserver or website with Javascript. Every language and that program requires resources to run. Let's focus on a Python webserver for this thought process.</p> <p>Our webserver (which is an application) requires a lot of things to be functional! It needs to access a network to open listening sockets, it needs access to a filesystem to read pages or write to a database (like sqlite). It needs CPU time to process requests, and memory to create a stack/heap to work through those requests. But as well our application also needs to be seperated and isolated from other programs too, so that they can not disclose our data - but so that faults in our application do not affect other services. It probably needs a seperate user and group, which is a key idea in unix process isolation and security. Maybe also there are things like SELinux or AppArmor that also provide extra enhancements.</p> <p>But why stop here there are many more. We might need system controls (sysctls) that define networking stack behaviour like how TCP performs. We may need specific versions of python libraries for our application. Perhaps we also want to limit the system calls that our python application can perform to our OS.</p> <p>I hope we can see that the resources we have, really is more than simply CPU and Memory here! Every application is really quite involved.</p> <h2 id="a-short-view-back-to-the-past">A short view back to the past ...</h2> <p>In the olden days, as developers we had to be responsible for these isolations. For example on a system, we'd have to select a bind address so that we could be configured to only use a single network device for listening on. This not only meant that our applications had to support this behaviour, but that a person had to read our documentation, and find out how to configure that behaviour to isolate the networking resource.</p> <p>And of course many others. To limit the amount of CPU or RAM that was available required you to configure ulimits for the user, and to select which user was going to run our application.</p> <p>Many problems have been seen too with a language like python where libraries are not isolated and there are conflicts between which <a href="/blog/html/2019/12/18/packaging_vendoring_and_how_it_s_changing.html">version different applications</a> require. Is it the fault of python? The application developer? It's hard to say ...</p> <p>What about system calls? With an interpretted language like python, you can't just set the capabilities flags or other hardening options because they have to be set on the interpretter (python) which is used in many places. An example where the resource (python) is shared between many applications preventing us from creating isolated python runtimes.</p> <p>Even things like SELinux and AppArmor required complex, hand created profiles, that were cryptic at best, or led to being disabled in the common case (It can't be secure if it's not usable! People will always take the easy option ...).</p> <p>And that's even before we look at init scripts - bash scripts that had to be invoked in careful ways, and were all hand rolled, each adding different mistakes or issues. It was a time where to &quot;package&quot; and application and deploy it, required huge amounts of knowledge of a broad range of topics.</p> <p>In many cases, I have seen another way this manifested. Rather than isolated applications (which was too hard), every application was installed on a dedicated virtual machine. The resource management then came as an artifact of every machine being seperate and managed by a hypervisor.</p> <h2 id="systemd">Systemd</h2> <p>Along came systemd though, and it got us much further. It provided consistent application launch tools, and has done a lot of work to manage and provide resource management such as cgroups (cpu, mem), dynamic users, some types of filesystem isolation and some more. Systemd as an init system has done some really good stuff.</p> <p>But still problems exist. Applications still require custom SELinux or AppArmor profiles, and systemd can't help managing network interfaces (that still falls on the application).</p> <p>It also still relies on you to put the files in place, or a distribution package to get the file content into the system.</p> <h2 id="docker">Docker</h2> <p>Docker takes this even further. Docker manages and arbitrates every resource your application requires, even the filesystem and install process. For example, a very complex topic like CPU or memory limit's on Linux, becomes quite simple in docker which exposes CPU and memory tunables. Docker allows sysctls per container. You assign and manage storage.</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>docker run -v db:/data/db -v config:/data/config --network private_net \ </span><span> --memory 1024M --shm-size 128M -P 80:8080 --user isolated \ </span><span> my/application:version </span></code></pre> <p>From this command we can see what resources are involved. We know that we mount two storage locations. We can see that we confine the network to a private network, and that we want to limit memory to 1024M. We also can see we'll be listening on port 80 which remaps to the container internally. We even know what user we'll run as so we can assign permissions to the volumes. Our application is also defined as is it's version.</p> <p>Not only can we see what resources we are using there are a lot of other benefits. Docker can dynamically generate selinux/apparmor isolation profiles, so we get stronger process isolation between containers and host processes. We know that the filesystem of this container is isolated from others, so they can have and bundle the correct versions of dependencies. We know how to start, stop, and even monitor the application. It can even have health checks. Logs will (should?) go to stdout/err which docker will forward to a log collector we can define. In the future each application may even be in it's own virtual memory space (IE seperate vm's).</p> <p>Docker provides a level of isolation to resources that is still hard to achieve in systemd, and not only that it makes very advanced or complex configurations easy to access and use. Accesibility of these features is vitally important to allow all people to create robust applications in their environments. But it also allows me as a developer to know what resources <em>can</em> exist in the container and how to interact with these in a way that will respect the wishes of the deploying administrator.</p> <h2 id="docker-isn-t-security-isolation">Docker Isn't Security Isolation</h2> <p>It's worth noting that while Docker can provide SELinux and AppArmor profiles, Docker is not an effective form of security isolation. It certainly makes the bar much much higher than before yes! And that's great! And I hope that bar continues to rise. However today we do live in an age where there are many attacks still locally on linux kernels and the fix delay in these is still long. We also still see CPU sidechannels, and these will never be resolved <a href="/blog/html/2020/01/20/there_are_no_root_causes.html">while we rely on asynchronous CPU behaviour</a>.</p> <p>If you have high value data, it is always best to have seperated physical machines for these applications, and to always patch frequently, have a proper CI/CD pipeline, and centralised logging, and much much more. Ask your security team! I'm sure they'd love to help :)</p> <h2 id="conclusion">Conclusion</h2> <p>For me personally, docker is about resource management and isolation. It helps me to define an interface that an admin can consume and interact with, making very advanced concepts easy to use. It gives me trust that applications will run in a way that is isolated and known all the way from development and testing through to production under high load. By making this accessible, it means that anyone - from a single docker container to a giant kubernetes cluster, can have really clear knowledge of how their applications are operating.</p> virt-manager missing pci.ids usb.ids macos Mon, 15 Jun 2020 00:00:00 +0000 Unknown https://fy.blackhats.net.au/blog/2020-06-15-virt-manager-missing-pci-ids-usb-ids-macos/ https://fy.blackhats.net.au/blog/2020-06-15-virt-manager-missing-pci-ids-usb-ids-macos/ <h1 id="virt-manager-missing-pci-ids-usb-ids-macos">virt-manager missing pci.ids usb.ids macos</h1> <p>I got the following error:</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>/usr/local/Cellar/libosinfo/1.8.0/share/libosinfo/pci.ids No such file or directory </span></code></pre> <p>This appears to be an issue in libosinfo from homebrew. Looking at the libosinfo source, there are some aux download files. You can fix this with:</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>mkdir -p /usr/local/Cellar/libosinfo/1.8.0/share/libosinfo/ </span><span>cd /usr/local/Cellar/libosinfo/1.8.0/share/libosinfo/ </span><span>wget -q -O pci.ids http://pciids.sourceforge.net/v2.2/pci.ids </span><span>wget -q -O usb.ids http://www.linux-usb.org/usb.ids </span></code></pre> <p>All is happy again with virt-manager</p> Resolving AirPlayXPCHelper Perr NULL kCanceledErr with Apple TV and MacOS Sun, 03 May 2020 00:00:00 +0000 Unknown https://fy.blackhats.net.au/blog/2020-05-03-resolving-airplayxpchelper-perr-null-kcancelederr-with-apple-tv-and-macos/ https://fy.blackhats.net.au/blog/2020-05-03-resolving-airplayxpchelper-perr-null-kcancelederr-with-apple-tv-and-macos/ <h1 id="resolving-airplayxpchelper-perr-null-kcancelederr-with-apple-tv-and-macos">Resolving AirPlayXPCHelper Perr NULL kCanceledErr with Apple TV and MacOS</h1> <p>I decided to finally get an Apple TV so that I could use my iPad and MacBook Pro to airplay to my projector. So far I've been really impressed by it and how well it works with modern amplifiers and my iPad.</p> <p>Sadly though, when I tried to use my MacBook pro to airplay to the Apple TV I recieved an &quot;Unable to connect&quot; error, with no further description.</p> <h2 id="initial-research">Initial Research</h2> <p>The first step was to look in console.app at the local system logs. The following item stood out:</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>error 09:24:41.459722+1000 AirPlayXPCHelper ### Error: CID 0xACF10006, Peer NULL, -6723/0xFFFFE5BD kCanceledErr </span></code></pre> <p>I only found a single result on a search for this, and they resolved the problem by disabling their MacOS firewall - attempting this myself did not fix the issue. There are also reports of apple service staff disabling the firewall to resolve airplay problems too.</p> <h2 id="time-to-dig-further">Time to Dig Further ...</h2> <p>Now it was time to look more. To debug an Apple TV you need to connect a USB-C cable to it's service port on the rear of the device, while you connect this to a Mac on the other side. Console.app will then show you the streamed logs from the device.</p> <p>While looking on the Apple TV I noticed the following log item:</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>[AirPlay] ### [0x8F37] Set up session 16845584210140482044 with [&lt;ipv6 address&gt;:3378]:52762 failed: 61/0x3D ECONNREFUSED { </span><span>&quot;timingProtocol&quot; : &quot;NTP&quot;, </span><span>&quot;osName&quot; : &quot;Mac OS X&quot;, </span><span>... </span><span>&quot;isScreenMirroringSession&quot; : true, </span><span>&quot;osVersion&quot; : &quot;10.15.4&quot;, </span><span>&quot;timingPort&quot; : 64880, </span><span>... </span><span>} </span></code></pre> <p>I have trimmed this log, as most details don't matter. What is important is that it looks like the Apple TV is attempting to back-connect to the MacBook Pro, which has a connection refused. From iOS it appears that the video/timing channel is initiated from the iOS device, so no back-connection is required, but for AirPlay to work from the MacBook Pro to the Apple TV, the Apple TV must be able to connect back on high ports with new UDP/TCP sessions for NTP to synchronise clocks.</p> <h2 id="my-network">My Network</h2> <p>My MacBook pro is on a seperate VLAN to my Apple TV for security reasons, mainly because I don't want most devices to access management consoles of various software that I have installed. I have used the Avahi reflector on my USG to enable cross VLAN discovery. This would appear to be issue, is that my firewall is not allowing the NTP traffic back to my MacBook pro.</p> <p>To resolve this I allowed some high ports from the Apple TV to connect back to the VLAN my MacBook Pro is on, and I allowed built-in software to recieve connections.</p> <p>Once this was done, I was able to AirPlay across VLANs to my Apple TV!</p> Building containers on OBS Mon, 20 Apr 2020 00:00:00 +0000 Unknown https://fy.blackhats.net.au/blog/2020-04-20-building-containers-on-obs/ https://fy.blackhats.net.au/blog/2020-04-20-building-containers-on-obs/ <h1 id="building-containers-on-obs">Building containers on OBS</h1> <p>My friend showed me how to build containers in OBS, the opensuse build service. It makes it really quite nice, as the service can parse your dockerfile, and automatically trigger rebuilds when any package dependency in the chain requires a rebuild.</p> <p>The simplest way is to have a seperate project for your containers to make the repository setup a little easier.</p> <p>When you edit the project metadata, if the project doesn't already exist, a new one is created. So we can start by filling out the template from the command:</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>osc meta prj -e home:firstyear:containers </span></code></pre> <p>This will give you a template: We need to add some repository lines:</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>&lt;project name=&quot;home:firstyear:apps&quot;&gt; </span><span> &lt;title&gt;Containers Demo&lt;/title&gt; </span><span> &lt;description&gt;Containers Demo&lt;/description&gt; </span><span> &lt;person userid=&quot;firstyear&quot; role=&quot;bugowner&quot;/&gt; </span><span> &lt;person userid=&quot;firstyear&quot; role=&quot;maintainer&quot;/&gt; </span><span> &lt;build&gt; </span><span> &lt;enable/&gt; </span><span> &lt;/build&gt; </span><span> &lt;publish&gt; </span><span> &lt;enable/&gt; </span><span> &lt;/publish&gt; </span><span> &lt;debuginfo&gt; </span><span> &lt;enable/&gt; </span><span> &lt;/debuginfo&gt; </span><span> &lt;!-- this repository --&gt; </span><span> &lt;repository name=&quot;containers&quot;&gt; </span><span> &lt;path project=&quot;openSUSE:Templates:Images:Tumbleweed&quot; repository=&quot;containers&quot;/&gt; </span><span> &lt;arch&gt;x86_64&lt;/arch&gt; </span><span> &lt;/repository&gt; </span><span>&lt;/project&gt; </span></code></pre> <p>Remember, to set the publist to &quot;enable&quot; if you want the docker images you build to be pushed to the registry!</p> <p>Now that that's done, we can check out the project, and create a new container package within.</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>osc co home:firstyear:containers </span><span>cd home:firstyear:containers </span><span>osc mkpac mycontainer </span></code></pre> <p>Now in the mycontainer folder you can start to build a container. Add your dockerfile:</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>#!BuildTag: mycontainer </span><span># </span><span># docker pull registry.opensuse.org/home/firstyear/apps/containers/mycontainer:latest </span><span># ^projectname ^repos ^ build tag </span><span>FROM opensuse/tumbleweed:latest </span><span> </span><span># </span><span># only one zypper ar command per line. only repositories inside the OBS are allowed </span><span># </span><span>RUN zypper ar http://download.opensuse.org/repositories/home:firstyear:apps/openSUSE_Tumbleweed/ &quot;home:firstyear:apps&quot; </span><span>RUN zypper mr -p 97 &quot;home:firstyear:apps&quot; </span><span>RUN zypper --gpg-auto-import-keys ref </span><span>RUN zypper install -y vim-data vim python3-ipython shadow python3-praw </span><span># Then the rest of your container as per usual ... </span></code></pre> <p>Then to finish up, you can commit this:</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>osc add Dockerfile </span><span>osc ci </span><span>osc results </span></code></pre> 389ds in containers Sat, 28 Mar 2020 00:00:00 +0000 Unknown https://fy.blackhats.net.au/blog/2020-03-28-389ds-in-containers/ https://fy.blackhats.net.au/blog/2020-03-28-389ds-in-containers/ <h1 id="389ds-in-containers">389ds in containers</h1> <p>I've spent a number of years working in the background to get 389-ds working in containers. I think it's very close to production ready (<a href="https://pagure.io/389-ds-base/issue/50989">one issue outstanding!</a>) and I'm now using it at home for my production LDAP needs.</p> <p>So here's a run down on using 389ds in a container!</p> <h2 id="getting-it-started">Getting it Started</h2> <p>The team provides an image for pre-release testing which you can get with docker pull:</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>docker pull 389ds/dirsrv:latest </span><span># OR, if you want to be pinned to the 1.4 release series. </span><span>docker pull 389ds/dirsrv:1.4 </span></code></pre> <p>The image can be run in an ephemeral mode (data will be lost on stop of the container) so you can test it:</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>docker run 389ds/dirsrv:1.4 </span></code></pre> <h2 id="making-it-persistent">Making it Persistent</h2> <p>To make your data persistent, you'll need to add a volume, and bind it to the container.</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>docker volume create 389ds </span></code></pre> <p>You can run 389ds where the container instance is removed each time the container stops, but the data persists (I promise this is safe!) with:</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>docker run --rm -v 389ds:/data -p 3636:3636 389ds/dirsrv:latest </span></code></pre> <p>Check your instance is working with an ldapsearch:</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>LDAPTLS_REQCERT=never ldapsearch -H ldaps://127.0.0.1:3636 -x -b &#39;&#39; -s base vendorVersion </span></code></pre> <p><em>NOTE: Setting the environment variable `LDAPTLS_REQCERT` to `never` disables CA verification of the LDAPS connection. Only use this in testing environments!</em></p> <p>If you want to make the container instance permanent (uses docker start/stop/restart etc) then you'll need to do a docker create with similar arguments:</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>docker create -v 389ds:/data -p 3636:3636 389ds/dirsrv:latest </span><span>docker ps -a </span><span>CONTAINER ID IMAGE ... NAMES </span><span>89b342c2e058 389ds/dirsrv:latest ... adoring_bartik </span></code></pre> <p>Remember, even if you rm the container instance, the volume stores all the data so you can re-pull the image and recreate the container and continue.</p> <h2 id="administering-the-instance">Administering the Instance</h2> <p>The best way is to the use the local LDAPI socket - by default the cn=Directory Manager password is randomised so that it can't be accessed remotely.</p> <p>To use the local LDAPI socket, you'll use docker exec into the running instance.</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>docker start &lt;container name&gt; </span><span>docker exec -i -t &lt;container name&gt; /usr/sbin/dsconf localhost &lt;cmd&gt; </span><span>docker exec -i -t &lt;container name&gt; /usr/sbin/dsconf localhost backend suffix list </span><span>No backends </span></code></pre> <p>In a container, the instance is always named &quot;localhost&quot;. So lets add a database backend now to our instance:</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>docker exec -i -t &lt;cn&gt; /usr/sbin/dsconf localhost backend create --suffix dc=example,dc=com --be-name userRoot </span><span>The database was sucessfully created </span></code></pre> <p>You can even go ahead and populate your backend now. To make it easier, specify your basedn into the volume's /data/config/container.inf. Once that's done we can setup sample data (including access controls), and create some users and groups.</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>docker exec -i -t &lt;cn&gt; /bin/sh -c &quot;echo -e &#39;\nbasedn = dc=example,dc=com&#39; &gt;&gt; /data/config/container.inf&quot; </span><span>docker exec -i -t &lt;cn&gt; /usr/sbin/dsidm localhost initialise </span><span>docker exec -i -t &lt;cn&gt; /usr/sbin/dsidm localhost user create --uid william --cn william \ </span><span> --displayName William --uidNumber 1000 --gidNumber 1000 --homeDirectory /home/william </span><span>docker exec -i -t &lt;cn&gt; /usr/sbin/dsidm localhost group create --cn test_group </span><span>docker exec -i -t &lt;cn&gt; /usr/sbin/dsidm localhost group add_member test_group uid=william,ou=people,dc=example,dc=com </span><span>docker exec -i -t &lt;cn&gt; /usr/sbin/dsidm localhost account reset_password uid=william,ou=people,dc=example,dc=com </span><span>LDAPTLS_REQCERT=never ldapwhoami -H ldaps://127.0.0.1:3636 -x -D uid=william,ou=people,dc=example,dc=com -W </span><span> Enter LDAP Password: </span><span> dn: uid=william,ou=people,dc=example,dc=com </span></code></pre> <p>There is much more you can do with these tools of course, but it's very easy to get started and working with an ldap server like this.</p> <h2 id="further-configuration">Further Configuration</h2> <p>Because this runs in a container, the approach to some configuration is a bit different. Some settings can be configured through either the content of the volume, or through environment variables.</p> <p>You can reset the directory manager password on startup but use the environment variable DS_DM_PASSWORD. Of course, please use a better password than &quot;password&quot;. pwgen is a good tool for this! This password persists across restarts, so you should make sure it's good.</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>docker run --rm -e DS_DM_PASSWORD=password -v 389ds:/data -p 3636:3636 389ds/dirsrv:latest </span><span>LDAPTLS_REQCERT=never ldapwhoami -H ldaps://127.0.0.1:3636 -x -D &#39;cn=Directory Manager&#39; -w password </span><span> dn: cn=directory manager </span></code></pre> <p>You can also configure certificates through pem files.</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>/data/tls/server.key </span><span>/data/tls/server.crt </span><span>/data/tls/ca/*.crt </span></code></pre> <p>All the certs in /data/tls/ca/ will be imported as CA's and the server key and crt will be used for the TLS server.</p> <p>If for some reason you need to reindex your db at startup, you can use:</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>docker run --rm -e DS_REINDEX=true -v 389ds:/data -p 3636:3636 389ds/dirsrv:latest </span></code></pre> <p>After the reindex is complete the instance will start like normal.</p> <h2 id="conclusion">Conclusion</h2> <p>389ds in a container is one of the easiest and quickest ways to get a working LDAP environment today. Please test it and let us know what you think!</p> APFS (why is df showing me funny numbers?!) Sat, 28 Mar 2020 00:00:00 +0000 Unknown https://fy.blackhats.net.au/blog/2020-03-28-apfs-why-is-df-showing-me-funny-numbers/ https://fy.blackhats.net.au/blog/2020-03-28-apfs-why-is-df-showing-me-funny-numbers/ <h1 id="apfs-why-is-df-showing-me-funny-numbers">APFS (why is df showing me funny numbers?!)</h1> <p>Apple's APFS has been the default for MacOS since High Sierra, where SSD (flash) automatically would convert from HFS+. This is a god send, especially with HFS+'s history of destroying any folder that has a large number of inodes within it.</p> <p>However, APFS behaves differently to previous filesystem technology. Let's see if we can explain why df reports multiple 932Gi disks like this:</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>&gt; df -h </span><span>Filesystem Size Used Avail Capacity iused ifree %iused Mounted on </span><span>/dev/disk1s5 932Gi 10Gi 380Gi 3% 484322 9767493838 0% / </span><span>/dev/disk1s1 932Gi 530Gi 380Gi 59% 2072480 9765905680 0% /System/Volumes/Data </span></code></pre> <p>And if we can explain why when you delete large files, you don't get any space back from df either.</p> <h2 id="how-it-looked-with-hfs">How it looked with HFS+</h2> <p>With HFS+ it was pretty simple. You had a disk (a block device), which had partitions (slices of the space in the block device) and those partitions were formatted with a filesystem that knew how to store data in them. An example:</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>&gt; diskutil list </span><span>... </span><span>/dev/disk2 (external, physical): </span><span> #: TYPE NAME SIZE IDENTIFIER </span><span> 0: GUID_partition_scheme *1.0 TB disk2 </span><span> 1: EFI EFI 209.7 MB disk2s1 </span><span> 2: Apple_HFS tmachine 999.9 GB disk2s2 </span></code></pre> <p>We can see that disk2 is 1.0TB in size, and it contains two partitions, the first is 209.7MB for EFI (disk2s1) and the second has data and formatted as HFS+ (disk2s2).</p> <p>Of course, this has some drawbacks - partitions don't like being moved, and filesystem resizing is a costly process of time and IO cycles. It's quite inflexible. If you wanted another partition here for read only data, well, you'd have to change a lot. Properties can only be applied to a filesystem as a whole, and they can't share space. If you had a 1TB drive partitioned to 500GB each, and were running low on space on one of them, well ... good luck! You have to move data manually, or change where applications store data.</p> <h2 id="apfs">APFS</h2> <p>APFS doesn't quite follow this model though. APFS is what's called a volume based filesystem. That means there is an intermediate layer in here. The layout looks like this in diskutil</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>&gt; diskutil list </span><span>... </span><span>/dev/disk0 (internal, physical): </span><span> #: TYPE NAME SIZE IDENTIFIER </span><span> 0: GUID_partition_scheme *1.0 TB disk0 </span><span> 1: EFI EFI 314.6 MB disk0s1 </span><span> 2: Apple_APFS Container disk1 1.0 TB disk0s2 </span></code></pre> <p>So our disk0 looks like before - an EFI partition, and a very large APFS container. However the container itself is NOT the filesystem. The contain is a pool of storage that APFS volumes are created into. We can see the volumes too.</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>&gt; diskutil list </span><span>... </span><span>/dev/disk1 (synthesized): </span><span> #: TYPE NAME SIZE IDENTIFIER </span><span> 0: APFS Container Scheme - +1.0 TB disk1 </span><span> Physical Store disk0s2 </span><span> 1: APFS Volume Macintosh HD — Data 569.4 GB disk1s1 </span><span> 2: APFS Volume Preboot 81.8 MB disk1s2 </span><span> 3: APFS Volume Recovery 526.6 MB disk1s3 </span><span> 4: APFS Volume VM 10.8 GB disk1s4 </span><span> 5: APFS Volume Macintosh HD 11.0 GB disk1s5 </span></code></pre> <p>Notice how /dev/disk1 is &quot;synthesized&quot;? It's not real - it's there to &quot;trick&quot; legacy tools into thinking that the container is a &quot;block&quot; device and the volumes are &quot;partitions&quot;.</p> <h2 id="benefits-of-volumes">Benefits of Volumes</h2> <p>One of the immediate benefits is that unlike partitions, in a volume filesystem, <em>all</em> the space of the underlying container (also known as: pool, volume group) is available to <em>all</em> volumes at anytime. Because the volumes are a flexible concept, they can have non-contiguous geometry on the disk (unlike a partition). That's why in your df output you can see:</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>&gt; df -h </span><span>Filesystem Size Used Avail Capacity iused ifree %iused Mounted on </span><span>/dev/disk1s5 932Gi 10Gi 380Gi 3% 484322 9767493838 0% / </span><span>/dev/disk1s1 932Gi 530Gi 380Gi 59% 2072480 9765905680 0% /System/Volumes/Data </span></code></pre> <p>Both disk1s5 (Macintosh HD) and disk1s1 (Macintosh HD --- Data) are APFS volumes. The container has 932Gi total space, and 380Gi available in the container which either volume could allocate. But you can also see the exact space reservation of each volume too: disk1s5 only has 10Gi in use, and disk1s1 has 530Gi in use.</p> <p>It would be very possible for disk1s1 to grow to fill all the space, and then to contract, and then have disk1s5 grow to take all the space and contract - this is because the space is flexibly allocated from the container. Neat!</p> <p>Each volume also can have different properties applied. For example, /dev/disk1s5 (Macintosh HD) in MacOS catalina is read-only:</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>/dev/disk1s5 on / (apfs, local, read-only, journaled) </span><span>/dev/disk1s1 on /System/Volumes/Data (apfs, local, journaled, nobrowse) </span></code></pre> <p>This is to prevent system tampering, and strengthen integrity of the system. There are a number of tricks to achieve this such as overlaying multiple volumes together. /Applications for example is actually a folder consitituted from the content of /System/Applications and /System/Volumes/Data/Applications. Anytime you &quot;drag&quot; and application to /Applications, you are actually putting it into /System/Volumes/Data/Applications. A very similar property holds for /Users (/System/Volumes/Data/Users), and even /Volumes.</p> <h2 id="copy-on-write-snapshots">Copy-on-Write, Snapshots</h2> <p>APFS is also a copy-on-write filesystem. This means whenever you write data, it's actually written to newly allocated disk regions, and the pointers are atomicly flipped to it. The full write occurs or it does not. This is part of the reason why APFS is so much better than HFS+ - in a crash your data is either in a previous state, or the new state - never a half written or corrupted state.</p> <p>This is the reason why APFS is only used on SSD (flash) devices - COW is very random IO write intensive, and on a rotational disk this would cause the head to &quot;seek&quot; randomly which would make both writes and reads very slow. SSD of course isn't affected by this, so having a highly fragmented file does not impose a penalty in the same way.</p> <p>Copy-on-Write however opens up some interesting behaviours. If you COW a file, but never remove the old version, you have a <em>snapshot</em>. This means you can have point-in-time views to how a filesystem was. This is actually used now by time machine during backups to ensure the content of a backup is stable before being written to the external backup media. It also allow time machine to perform &quot;backups&quot; while you are out-and-about, by snapshotting as you work. Because snapshots are just &quot;not removing old data&quot; they are low overhead to maintain and take snapshots.</p> <p>You can see snapshots on your system with:</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>&gt; tmutil listlocalsnapshots / </span><span>Snapshots for volume group containing disk /: </span><span>com.apple.TimeMachine.2020-03-27-084939.local </span><span>com.apple.TimeMachine.2020-03-27-100157.local </span><span>com.apple.TimeMachine.2020-03-27-105937.local </span><span>com.apple.TimeMachine.2020-03-27-121414.local </span><span>... </span></code></pre> <p>You can even take your own snapshots if you want!</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>&gt; time tmutil localsnapshot </span><span>Created local snapshot with date: 2020-03-28-091943 </span><span>tmutil localsnapshot 0.01s user 0.01s system 4% cpu 0.439 total </span></code></pre> <p>See how fast that is! Remember also because this is based on copy-on-write, the snapshots only take as much data as the <em>differences</em>, or what you are changing as you work.</p> <h2 id="space-reclaim">Space Reclaim</h2> <p>This leads to the final point of confusion - when people delete files to clear space, but df reports no change. For example:</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>&gt; df -h </span><span>Filesystem Size Used Avail Capacity iused ifree %iused Mounted on </span><span>/dev/disk1s1 932Gi 530Gi 380Gi 59% 2072480 9765905680 0% /System/Volumes/Data </span><span>&gt; ls -alh Downloads/Windows_Server_2016_Datacenter_EVAL_en-us_14393_refresh.ISO </span><span>-rwx------@ 1 william staff 6.5G 10 Oct 2018 Downloads/Windows_Server_2016_Datacenter_EVAL_en-us_14393_refresh.ISO </span><span>&gt; rm Downloads/Windows_Server_2016_Datacenter_EVAL_en-us_14393_refresh.ISO </span><span>&gt; df -h </span><span>Filesystem Size Used Avail Capacity iused ifree %iused Mounted on </span><span>/dev/disk1s1 932Gi 530Gi 380Gi 59% 2072479 9765905681 0% /System/Volumes/Data </span></code></pre> <p>Now I promise, I really did delete the file - check the &quot;iused&quot; and &quot;ifree&quot; columns. But also note that the &quot;Used&quot; space didn't change? Surely we should expect to see this value drop to 523Gi since I removed a 6.5G file.</p> <p>Remember that APFS is a voluming filesystem, with copy-on-write. Due to snapshots, the space used in a volume is the sum of active data <em>and</em> snapshotted data. This means that when you are removing a file you are removing it from the volume at this point in time, but it may still exist in snapshots that exist in the volume! That's why there is a reduction in the iused/ifree (an inode pointer was removed) but no change in the space (the file still exists in a snapshot).</p> <p>During normal operation, provided there is sufficent freespace, you won't actually notice this behaviour. But when you say ... have not a lot of space left (maybe 10G), and you delete some files to import something (say a 40G import), you try the copy again ... and it fails! Drat! But you wait a bit and suddenly it works? What in heck happened?</p> <p>In the background, MacOS has registered &quot;okay, the user demands at least 30G more space to complete this task. Let's clean snapshots until we have that much space available&quot;. The snapshots are pruned so when you come back later, suddenly you have the space.</p> <p>Again, you can actually do this yourself. tmutil has a command &quot;thinlocalsnapshots&quot; for this. An example usage would be:</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>&gt; tmutil thinlocalsnapshots /System/Volumes/Data [bytes required] </span><span>Thinned local snapshots: </span></code></pre> <p>In my case I have a lot of space available, so no snapshots are pruned. But you may find that multiple snapshots are removed in this process!</p> <h2 id="conclusion">Conclusion</h2> <p>APFS is actually a really cool piece of filesystem technology, and I think has made MacOS one of the most viable platforms for reliable daily use. It embraces many great ideas, and despite it's youth, has done really well. But those new ideas conflict with legacy, and have some behaviours that are not always clearly exposed on shown to you, the user. Understanding those behaviours means we can see <em>why</em> our computers are behaving in certain - sometimes unexpected - ways.</p> USG fixing avahi Sun, 15 Mar 2020 00:00:00 +0000 Unknown https://fy.blackhats.net.au/blog/2020-03-15-usg-fixing-some-basic-issues/ https://fy.blackhats.net.au/blog/2020-03-15-usg-fixing-some-basic-issues/ <h1 id="usg-fixing-avahi">USG fixing avahi</h1> <p>Sadly on the USG pro 4 avahi will regularly spiral out of control taking up 100% cpu. To fix this, we set an hourly restart:</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>sudo -s </span><span>crontab -e </span></code></pre> <p>Then add:</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>15 * * * * /usr/sbin/service avahi-daemon restart </span></code></pre> Fedora 32 Wallpaper Submission - Story Sat, 14 Mar 2020 00:00:00 +0000 Unknown https://fy.blackhats.net.au/blog/2020-03-14-fedora-32-wallpaper-submission-story/ https://fy.blackhats.net.au/blog/2020-03-14-fedora-32-wallpaper-submission-story/ <h1 id="fedora-32-wallpaper-submission-story">Fedora 32 Wallpaper Submission - Story</h1> <p>Fedora opens submissions for <a href="https://apps.fedoraproject.org/nuancier/contribute/">wallpapers</a> to be submitted for the next version of the release. I used fedora for a long time, so I decided to submit this photo, and write this post to talk about it:</p> <p><img src="/_static/20191119_184819_DSCF0043_5.jpg" alt="image" /></p> <p>This was takeing on 2019-11-19 in my home city of Adelaide, South Australia. I had traveled to see some friends over Christmas. We went to <a href="https://www.google.com/maps/@-34.9656653,138.6670176,14.51z">Mount Osmond</a> to take some photos, and I took this as we walked up to the lookout.</p> <p>The next day, this area was a high risk location for a possible bushfire - and many bushfires have since devastated many regions of Australia, affecting many people that I know.</p> <p>I really find that the Australian landscape is so different to Europe or Asia - many tones of subtle reds, browns, and more. A dry and dusty look. The palette is such a contrast to the lush greens of Europe. Australia is a really beautiful country, in a very distinct and striking manner.</p> <p>Anyway, I hope you like the photo :)</p> Fixing a MacBook Pro 8,2 with dead AMD GPU Tue, 04 Feb 2020 00:00:00 +0000 Unknown https://fy.blackhats.net.au/blog/2020-02-04-fixing-a-macbook-pro-8-2-with-dead-amd-gpu/ https://fy.blackhats.net.au/blog/2020-02-04-fixing-a-macbook-pro-8-2-with-dead-amd-gpu/ <h1 id="fixing-a-macbook-pro-8-2-with-dead-amd-gpu">Fixing a MacBook Pro 8,2 with dead AMD GPU</h1> <p>I've owned a MacBook Pro 8,2 late 2011 edition, which I used from 2011 to about 2018. It was a great piece of hardware, and honestly I'm surprised it lasted so long given how many MacOS and Fedora installs it's seen.</p> <p>I upgraded to a MacBook Pro 15,1, and I gave the 8,2 to a friend who was in need of a new computer so she could do her work. It worked really well for her until today when she's messaged me that the machine is having a problem.</p> <h2 id="the-problem">The Problem</h2> <p>The machine appeared to be in a bootloop, where just before swapping from the EFI GPU to the main display server, it would go black and then lock up/reboot. Booting to single user mode (boot holding cmd + s) showed the machine's disk was intact with a clean apfs. The system.log showed corruption at the time of the fault, which didn't instill confidence in me.</p> <p>Attempting a recovery boot (boot holding cmd + r), this also yielded the bootloop. So we have potentially eliminated the installed copy of MacOS as the source of the issue.</p> <p>I've then used the apple hardware test (boot while holding d), and it has passed the machine as a clear bill of health.</p> <p>I have seen one of these machines give up in the past - my friends mother had one from the same generation and that died in almost the same way - could it be the same?</p> <h2 id="the-8-2-s-cursed-gpu-stack">The 8,2's cursed gpu stack</h2> <p>The 8,2 15&quot; mbp has dual gpu's - it has the on cpu Intel 3000, and an AMD radeon 6750M. The two pass through an LVDS graphics multiplexer to the main panel. The external display port however is not so clear - the DDC lines are passed through the GMUX, but the datalines directly attach to the the display port.</p> <p>The machine is also able to boot with EFI rendering to either card. By default this is the AMD radeon. Which ever card is used at boot is also the first card MacOS attempts to use, but it will try to swap to the radeon later on.</p> <p>This generation had a large number of the radeons develop faults in their 3d rendering capability so it would render the EFI buffer correctly, but on the initiation of 3d rendering it would fail. Sounds like what we have here!</p> <h2 id="to-fix-this">To fix this ...</h2> <p>Okay, so this is fixable. First, we need to tell EFI to boot primarily from the intel card. Boot to single user mode and then run.</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>nvram fa4ce28d-b62f-4c99-9cc3-6815686e30f9:gpu-power-prefs=%01%00%00%00 </span></code></pre> <p>Now we need to prevent loading of the AMD drivers so that during the boot MacOS doesn't attempt to swap from Intel to the Radeon. We can do this by hiding the drivers. System integrity protection will stop you, so you need to do this as part of recovery. Boot with cmd + r, which now works thanks to the EFI changes, then open terminal</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>cd /Volumes/Macintosh HD </span><span>sudo mkdir amdkext </span><span>sudo mv System/Library/Extensions/AMDRadeonX3000.kext amdkext/ </span></code></pre> <p>Then reboot. You'll notice the fans go crazy because the Radeon card can't be disabled without the driver. We can post-boot load the driver to stop the fans to fix this up.</p> <p>To achieve this we make a helper script:</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span># cat /usr/local/libexec/amd_kext_load.sh </span><span>#!/bin/sh </span><span>/sbin/kextload /amdkext/AMDRadeonX3000.kext </span></code></pre> <p>And a launchctl daemon</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span># cat /Library/LaunchDaemons/au.net.blackhats.fy.amdkext.plist </span><span>&lt;?xml version=&quot;1.0&quot; encoding=&quot;UTF-8&quot;?&gt; </span><span>&lt;!DOCTYPE plist PUBLIC &quot;-//Apple//DTD PLIST 1.0//EN&quot; &quot;http://www.apple.com/DTDs/PropertyList-1.0.dtd&quot;&gt; </span><span>&lt;plist version=&quot;1.0&quot;&gt; </span><span> &lt;dict&gt; </span><span> &lt;key&gt;Label&lt;/key&gt; </span><span> &lt;string&gt;au.net.blackhats.fy.amdkext&lt;/string&gt; </span><span> &lt;key&gt;Program&lt;/key&gt; </span><span> &lt;string&gt;/usr/local/libexec/amd_kext_load.sh&lt;/string&gt; </span><span> &lt;key&gt;RunAtLoad&lt;/key&gt; </span><span> &lt;true/&gt; </span><span> &lt;key&gt;StandardOutPath&lt;/key&gt; </span><span> &lt;string&gt;/var/log/amd_kext_load.log&lt;/string&gt; </span><span> &lt;/dict&gt; </span><span>&lt;/plist&gt; </span></code></pre> <p>Now if you reboot, you'll have a working mac, and the fans will stop properly. I've tested this with suspend and resume too and it works! The old beast continues to live :)</p> There are no root causes Mon, 20 Jan 2020 00:00:00 +0000 Unknown https://fy.blackhats.net.au/blog/2020-01-20-there-are-no-root-causes/ https://fy.blackhats.net.au/blog/2020-01-20-there-are-no-root-causes/ <h1 id="there-are-no-root-causes">There are no root causes</h1> <p>At Gold Coast LCA2020 I gave a <a href="https://www.youtube.com/watch?v=eqQUepwTHjA&amp;t=25m47s">lightning talk on swiss cheese</a>. Well, maybe not really swiss cheese. But it was about the <a href="https://en.wikipedia.org/wiki/Swiss_cheese_model">swiss cheese failure model</a> which was proposed at the university of manchester.</p> <p>Please note this will cover some of the same topics as the talk, but in more detail, and with less jokes.</p> <h2 id="an-example-problem">An example problem</h2> <p>So we'll discuss the current issues behind modern CPU isolation attacks IE spectre. Spectre is an attack that uses timing of a CPU's speculative execution unit to retrieve information from another running process on the same physical system.</p> <p>Modern computers rely on hardware features in their CPU to isolate programs from each other. This could be isolating your web-browser from your slack client, or your sibling's login from yours.</p> <p>This isolation however has been compromised by attacks like Spectre, and it looks unlikely that it can be resolved.</p> <h2 id="what-is-speculative-execution">What is speculative execution?</h2> <p>In order to be &quot;fast&quot; modern CPU's are far more complex than most of us have been taught. Often we believe that a CPU thread/core is executing &quot;one instruction/operation&quot; at a time. However this isn't how most CPU's work. Most work by having a pipeline of instructions that are in various stages of execution. You could imagine it like this:</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>let mut x = 0 </span><span>let mut y = 0 </span><span>x = 15 * some_input; </span><span>y = 10 * other_input; </span><span>if x &gt; y { </span><span> return true; </span><span>} else { </span><span> return false; </span><span>} </span></code></pre> <p>This is some made up code, but in a CPU, every part of this could be in the &quot;pipeline&quot; at once.</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>let mut x = 0 &lt;&lt;-- at the head of the queue and &quot;further&quot; along completion </span><span>let mut y = 0 &lt;&lt;-- it&#39;s executed part way, but not to completion </span><span>x = 15 * some_input; </span><span>y = 10 * other_input; &lt;&lt;-- all of these are in pipeline, and partially complete </span><span>if x &gt; y { &lt;&lt;-- what happens here? </span><span> return true; </span><span>} else { </span><span> return false; </span><span>} </span></code></pre> <p>So how does this &quot;pipeline&quot; handle the if statement? If the pipeline is looking ahead, how can we handle a choice like an if? Can we really predict the future?</p> <h2 id="speculative-execution">Speculative execution</h2> <p>At the if statement, the CPU uses past measurements to make a <em>prediction</em> about which branch <em>might</em> be taken, and it then begins to execute that path, even though 'x &gt; y' has not been executed or completed yet! At this point x or y may not have even finished <em>being computed</em> yet!</p> <p>Let's assume for now our branch predictor thinks that 'x &gt; y' is false, so we'll start to execute the &quot;return false&quot; or any other content in that branch.</p> <p>Now the instructions ahead catch up, and we resolve &quot;did we really predict correctly?&quot;. If we did, great! We have been able to advance the program state <em>asynchronously</em> even without knowing the answer until we get there.</p> <p>If not, ohh nooo. We have to unwind what we were doing, clear some of the pipeline and try to do the correct branch.</p> <p>Of course this has an impact on <em>timing</em> of the program. Some people found you could write a program to manipulate this predictor and using specific addresses and content, they could use these timing variations to &quot;access memory&quot; they are not allowed to by letting the specualative executor contribute to code they are not allowed to access before the unroll occurs. They could time this, and retrieve the memory contents from areas they are not allowed to access, breaking isolation.</p> <h2 id="owwww-my-brain">Owwww my brain</h2> <p>Yes. Mine too.</p> <h2 id="community-reactions">Community Reactions</h2> <p>Since this has been found, a large amount of the community reaction has been about the &quot;root cause&quot;. 'Clearly' the root cause is &quot;Intel are bad at making CPU's&quot; and so everyone should buy AMD instead because they &quot;weren't affected quite as badly&quot; (Narrators voice: <a href="https://www.zdnet.com/article/amd-processors-from-2011-to-2019-vulnerable-to-two-new-attacks/">They were absolutely just as bad</a>). We've had some intel CPU updates and kernel/program fixes so all good right? We addressed the root cause.</p> <h2 id="or-did-we">Or ... did we?</h2> <p>Our computers are still asynchronous, and contain many out-of-order parts. It's hard to believe we have &quot;found&quot; every method of exploiting this. Indeed in the last year many more ways to bypass hardware isolation due to our systems async nature have been found.</p> <p>Maybe the &quot;root cause&quot; wasn't addressed. Maybe ... there are no ....</p> <h2 id="history">History</h2> <p>To understand how we got to this situation we need to look at how CPU's have evolved. This is not a complete history.</p> <p>The PDP11 was a system owned at bell labs, where the C programing language was developed. Back then CPU's were very simple - A CPU and memory, executing one instruction at a time.</p> <p>The C programming language gained a lot of popularity as it was able to be &quot;quickly&quot; ported to other CPU models to allow software to be compiled on other platforms. This led to many systems being developed in C.</p> <p>Intel introduced the 8086, and many C programs were ported to run on it. Intel then released the 80486 in 1989, which had the first pipeline and cache to improve performance. In order to continue to support C, this meant the memory model could not change from the PDP11 - the cache had to be transparent, and the pipeline could not expose state.</p> <p>This has of course led to computers being more important in our lives and businesses, so we expected further performance, leading to increased frequencies and async behaviours.</p> <p>The limits of frequencies were really hit in the Pentium 4 era, when about 4GHz was shown to be a barrier of stability for those systems. They had very deep pipelines to improve performance, but that also had issues when branch prediction failed causing pipeline stalls. Systems had to improve their async behaviours <em>futher</em> to squeeze every single piece of performance possible out.</p> <p>Compiler developers also wanted more performance so they started to develop ways to transform C in ways that &quot;took advantage&quot; of x86_64 tricks, by manipulating the environment so the CPU is &quot;hinted&quot; into states we &quot;hope&quot; it gets into.</p> <p>Many businesses also started to run servers to provide to consumers, and in order to keep costs low they would put many users onto single pieces of hardware so they could share or overcommit resources.</p> <p>This has created a series of positive reinforcement loops - C is 'abi stable' so we keep developing it due to it's universal nature. C code can't be changed without breaking every existing system. We can't change the CPU memory model without breaking C, which is hugely prevalent. We improve the CPU to make C faster, transparently so that users/businesses can run more C programs and users. And then we improve compilers to make C faster given quirks of the current CPU models that exist ...</p> <h2 id="swiss-cheese-model">Swiss cheese model</h2> <p>It's hard to look at the current state of systems security and simply say &quot;it's the cpu vendors fault&quot;. There are many layers that have come together to cause this situation.</p> <p>This is called the &quot;swiss cheese model&quot;. Imagine you take a stack of swiss cheese and rotate and rearrange the slices. You will not be able to see through it. but as you continue to rotate and rearrange, eventually you may see a tunnel through the cheese where all the holes line up.</p> <p>This is what has happened here - we developed many layers socially and technically that all seemed reasonable over time, and only after enough time and re-arrangements of the layers, have we now arrived at a situation where a failure has occured that permeates all of computer hardware.</p> <p>To address it, we need to look beyond just &quot;blaming hardware makers&quot; or &quot;software patches&quot;. We need to help developers move away from C to other languages that can be brought onto new memory models that have manual or other cache strategies. We need hardware vendors to implement different async models. We need to educate businesses on risk analysis and how hardware works to provide proper decision making capability. We need developers to alter there behaviour to work in environments with higher performance constraints. And probably much much more.</p> <h2 id="there-are-no-root-causes-1">There are no root causes</h2> <p>It is a very pervasive attitude in IT that every issue has a root cause. However, looking above we can see it's never quite so simple.</p> <p>Saying an issue has a root cause, prevents us from examining the social, political, economic and human factors that all become contributing factors to failure. Because we are unable to examine them, we are unable to address the various layers that have contributed to our failures.</p> <p>There are no root causes. Only contributing factors.</p> Concurrency 1: Types of Concurrency Sun, 29 Dec 2019 00:00:00 +0000 Unknown https://fy.blackhats.net.au/blog/2019-12-29-concurrency-1-types-of-concurrency/ https://fy.blackhats.net.au/blog/2019-12-29-concurrency-1-types-of-concurrency/ <h1 id="concurrency-1-types-of-concurrency">Concurrency 1: Types of Concurrency</h1> <p>I want to explain different types of concurrent datastructures, so that we can explore their properties and when or why they might be useful.</p> <p>As our computer systems become increasingly parallel and asynchronous, it's important that our applications are able to work in these environments effectively. Languages like Rust help us to ensure our concurrent structures are safe.</p> <h2 id="cpu-memory-model-crash-course">CPU Memory Model Crash Course</h2> <p>In no way is this a thorough, complete, or 100% accurate representation of CPU memory. My goal is to give you a quick brief on how it works. I highly recommend you read <a href="https://people.freebsd.org/~lstewart/articles/cpumemory.pdf">&quot;what every programmer should know about memory&quot;</a> if you want to learn more.</p> <p>In a CPU we have a view of a memory space. That could be in the order of KB to TB. But it's a single coherent view of that space.</p> <p>Of course, over time systems and people have demanded more and more performance. But we also have languages like C, that won't change from their view of a system as a single memory space, or change how they work. Of course, it turns out <a href="https://queue.acm.org/detail.cfm?id=3212479">C is not a low level language</a> but we like to convince ourselves it is.</p> <p>To keep working with C and others, CPU's have acquired cache's that are transparent to the operation of the memory. You have no control of what is - or is not - in the cache. It &quot;just happens&quot; asynchronously. This is exactly why spectre and meltdown happened (and will continue to happen) because these async behaviours will always have the observable effect of making your CPU faster. Who knew!</p> <p>Anyway, for this to work, each CPU has multiple layers of cache. At L3 the cache is shared with all the cores on the die. At L1 it is &quot;per cpu&quot;.</p> <p>Of course it's a single view into memory. So if address 0xff is in the CPU cache of core 1, and also in cache of core 2, what happens? Well it's supported! Caches between cores are kept in sync via a state machine called MESI. These states are:</p> <ul> <li>Exclusive - The cache is the only owner of this value, and it is unchanged.</li> <li>Modified - The cache is the only owner of this value, and it has been changed.</li> <li>Invalid - The cache holds this value but another cache has changed it.</li> <li>Shared - This cache and maybe others are viewing this valid value.</li> </ul> <p>To gloss very heavily over this topic, we want to avoid invaild. Why? That means two cpus are <em>contending</em> for the value, causing many attempts to keep each other in check. These contentions cause CPU's to slow down.</p> <p>We want values to either be in E/M or S. In shared, many cpu's are able to read the value at maximum speed, all the time. In E/M, we know only this cpu is changing the value.</p> <p>This cache coherency is also why mutexes and locks exist - they issue the needed CPU commands to keep the caches in the correct states for the memory we are accessing.</p> <p>Keep in mind Rust's variables are immutable, and able to share between threads, or mutable and single thread only. Sound familar? Rust is helping with concurrency by keeping our variables in the fastest possible cache states.</p> <h2 id="data-structures">Data Structures</h2> <p>We use data structures in programming to help improve behaviours of certain tasks. Maybe we need to find values quicker, sort contents, or search for things. Data Structures are a key element of modern computer performance.</p> <p>However most data structures are not thread safe. This means only a single CPU can access or change them at a time. Why? Because if a second read them, due to cache-differences in content the second CPU may see an invalid datastructure, leading to undefined behaviour.</p> <p>Mutexes can be used, but this causes other CPU's to stall and wait for the mutex to be released -not really what we want on our system. We want every CPU to be able to process data without stopping!</p> <h2 id="thread-safe-datastructures">Thread Safe Datastructures</h2> <p>There exist many types of thread safe datastructures that can work on parallel systems. They often avoid mutexes to try and keep CPU's moving as fast as possible, relying on special atomic cpu operations to keep all the threads in sync.</p> <p>Multiple classes of these structures exist, which have different properties.</p> <h2 id="mutex">Mutex</h2> <p>I have mentioned these already, but it's worth specifying the properties of a mutex. A mutex is a system where a single CPU exists in the mutex. It becomes one &quot;reader/writer&quot; and all other CPU's must wait until the mutex is released by the current CPU holder.</p> <h2 id="read-write-lock">Read Write Lock</h2> <p>Often called RWlock, these allow one writer OR multiple parallel readers. If a reader is reading then a writer request is delayed until the readers complete. If a writer is changing data, all new reads are blocked. All readers will always be reading the same data.</p> <p>These are great for highly concurrent systems provided your data changes infrequently. If you have a writer changing data a lot, this causes your readers to be continually blocking. The delay on the writer is also high due to a potentially high amount of parallel readers that need to exit.</p> <h2 id="lock-free">Lock Free</h2> <p>Lock free is a common (and popular) datastructue type. These are structures that don't use a mutex at all, and can have multiple readers <em>and</em> multiple writers at the same time.</p> <p>The most common and popular structure for lock free is queues, where many CPUs can append items and many can dequeue at the same time. There are also a number of lock free sets which can be updated in the same way.</p> <p>An interesting part of lock free is that all CPU's are working on the same set - if CPU 1 reads a value, then CPU 2 writes the same value, the next read from CPU 1 will show the new value. This is because these structures aren't transactional - lock free, but not transactional. There are some times where this is really useful as a property when you need a single view of the world between all threads, and your program can tolerate data changing between reads.</p> <h2 id="wait-free">Wait Free</h2> <p>This is a specialisation of lock free, where the reader/writer has guaranteed characteristics about the time they will wait to read or write data. This is very detailed and subtle, only affecting real time systems that have strict deadline and performance requirements.</p> <h2 id="concurrently-readable">Concurrently Readable</h2> <p>In between all of these is a type of structure called concurrently readable. A concurrently readable structure allows one writer <em>and</em> multiple parallel readers. An interesting property is that when the reader &quot;begins&quot; to read, the view for that reader is guaranteed not to change until the reader completes. This means that the structure is transactional.</p> <p>An example being if CPU 1 reads a value, and CPU 2 writes to it, CPU 1 would NOT see the change from CPU 2 - it's outside of the read transaction!</p> <p>In this way there are a lot of read-only immutable data, and one writer mutating and changing things ... sounds familar? It's very close to how our CPU's cache work!</p> <p>These structures also naturally lend themself well to long processing or database systems where you need transactional (ACID) properties. In fact some databases use concurrent readable structures to achieve ACID semantics.</p> <p>If it's not obvious - concurrent readability is where my interest lies, and in the next post I'll discuss some specific concurrently readable structures that exist today, and ideas for future structures.</p> Concurrency 2: Concurrently Readable Structures Sun, 29 Dec 2019 00:00:00 +0000 Unknown https://fy.blackhats.net.au/blog/2019-12-29-concurrency-2-concurrently-readable-structures/ https://fy.blackhats.net.au/blog/2019-12-29-concurrency-2-concurrently-readable-structures/ <h1 id="concurrency-2-concurrently-readable-structures">Concurrency 2: Concurrently Readable Structures</h1> <p>In this post, I'll discuss concurrently readable datastructures that exist, and ideas for future structures. Please note, this post is an inprogress design, and may be altered in the future.</p> <p>Before you start, make sure you have <a href="../concurrency_1_types_of_concurrency.html">read part 1</a></p> <h2 id="concurrent-cell">Concurrent Cell</h2> <p>The simplest form of concurrently readable structure is a concurrent cell. This is equivalent to a read-write lock, but has concurrently readable properties instead. The key mechanism to enable this is that when the writer begins, it clones the data before writing it. We trade more memory usage for a gain in concurrency.</p> <p>To see an implementation, see my <a href="https://crates.io/crates/concread">rust crate, concread</a></p> <h2 id="concurrent-tree">Concurrent Tree</h2> <p>The concurrent cell is good for small data, but a larger structure - like a tree - may take too long to clone on each write. A good estimate is that if your data in the cell is larger than about 512 bytes, you likely want a concurrent tree instead.</p> <p>In a concurrent tree, only the <em>branches</em> involved in the operation are cloned. Imagine the following tree:</p> <p><img src="/_static/cow_1.png" alt="image" /></p> <p>When we attempt to change a value in the 4th leaf we copy it before we begin, and all it's parents to update their pointers.</p> <p><img src="/_static/cow_2.png" alt="image" /></p> <p>In the process the pointers from the new root b to branch 1 are maintained. The new second branch also maintains a pointer to the original 3rd leaf.</p> <p>This means that in this example only 3/7 nodes are copied, saving a lot of cloning. As your tree grows this saves a lot of work. Consider a tree with node-widths of 7 pointers and at height level 5. Assuming perfect layout, you only need to clone 5/~16000 nodes. A huge saving in memory copy!</p> <p>The interesting part is a reader of root a, also is unaffected by the changes to root b - the tree from root a hasn't been changed, as all it's pointers and nodes are still valid.</p> <p>When all readers of root a end, we clean up all the nodes it pointed to that no longer are needed by root b (this can be done with atomic reference counting, or garbage lists in transactions).</p> <p><img src="/_static/cow_3.png" alt="image" /></p> <p>It is through this copy-on-write (also called multi view concurrency control) that we achieve concurrent readability in the tree.</p> <p>This is really excellent for databases where you have in memory structures that work in parallel to the database transactions. In kanidm an example is the in-memory schema that is used at run time but loaded from the database. They require transactional behaviours to match the database, and ACID properties so that readers of a past transaction have the &quot;matched&quot; schema in memory.</p> <h2 id="concurrent-cache-updated-2020-05-13">Concurrent Cache (Updated 2020-05-13)</h2> <p>A design that I have thought about for a long time has finally come to reality. This is a concurrently readable transactional cache. One writer, multiple readers with consistent views of the data. Additionally due to the transactioal nature, rollbacks and commits are fulled supported.</p> <p>For a more formal version of this design, please see <a href="https://github.com/Firstyear/concread/blob/master/CACHE.md">my concurrent ARC draft paper</a>.</p> <p>This scheme should work with any cache type - LRU, LRU2Q, LFU. I have used <a href="https://web.archive.org/web/20100329071954/http://www.almaden.ibm.com/StorageSystems/projects/arc/">ARC</a>.</p> <p>ARC was popularised by ZFS - ARC is not specific to ZFS, it's a strategy for cache replacement, despite the comment association between the two.</p> <p>ARC is a pair of LRU's with a set of ghost lists and a weighting factor. When an entry is &quot;missed&quot; it's inserted to the &quot;recent&quot; LRU. When it's accessed from the LRU a second time, it moves to the &quot;frequent&quot; LRU.</p> <p>When entries are evicted from their sets they are added to the ghost list. When a cache miss occurs, the ghost list is consulted. If the entry &quot;would have been&quot; in the &quot;recent&quot; LRU, but was not, the &quot;recent&quot; LRU grows and the &quot;frequent&quot; LRU shrinks. If the item &quot;would have been&quot; in the &quot;frequent&quot; LRU but was not, the &quot;frequent&quot; LRU is expanded, and the &quot;recent&quot; LRU shrunk.</p> <p>This causes ARC to be self tuning to your workload, as well as balancing &quot;high frequency&quot; and &quot;high locality&quot; operations. It's also resistant to many cache invalidation or busting patterns that can occur in other algorithms.</p> <p>A major problem though is ARC is not designed for concurrency - LRU's rely on double linked lists which is <em>very</em> much something that only a single thread can modify safely due to the number of pointers that are not aligned in a single cache line, prevent atomic changes.</p> <h2 id="how-to-make-arc-concurrent">How to make ARC concurrent</h2> <p>To make this concurrent, I think it's important to specify the goals.</p> <ul> <li>Readers must always have a correct &quot;point in time&quot; view of the cache and its data</li> <li>Readers must be able to trigger cache inclusions</li> <li>Readers must be able to track cache hits accurately</li> <li>Readers are isolated from all other readers and writer actions</li> <li>Writers must always have a correct &quot;point in time&quot; view</li> <li>Writers must be able to rollback changes without penalty</li> <li>Writers must be able to trigger cache inclusions</li> <li>Writers must be able to track cache hits accurately</li> <li>Writers are isolated from all readers</li> <li>The cache must maintain correct temporal ordering of items in the cache</li> <li>The cache must properly update hit and inclusions based on readers and writers</li> <li>The cache must provide ARC semantics for management of items</li> <li>The cache must be concurrently readable and transactional</li> <li>The overhead compared to single thread ARC is minimal</li> </ul> <p>There are a lot of places to draw inspiration from, and I don't think I can list - or remember them all.</p> <p>My current design uses a per-thread reader cache to allow inclusions, with a channel to asynchronously include and track hits to the write thread. The writer also maintains a local cache of items including markers of removed items. When the writer commits, the channel is drained to a time point T, and actions on the ARC taken.</p> <p>This means the LRU's are maintained only in a single write thread, but the readers changes are able to influence the caching decisions.</p> <p>To maintain consistency, and extra set is added which is the haunted set, so that a key that has existed at some point can be tracked to identify it's point in time of eviction and last update so that stale data can never be included by accident.</p> <h2 id="limitations-and-concerns">Limitations and Concerns</h2> <p>Cache missing is very expensive - multiple threads may load the value, the readers must queue the value, and the writer must then act on the queue. Sizing the cache to be large enough is critically important as eviction/missing will have a higher penalty than normal. Optimally the cache will be &quot;as large or larger&quot; than the working set.</p> <p>But with a concurrent ARC we now have a cache where each reader thread has a thread local cache and the writer is communicated to by channels. This may make the cache's memory limit baloon to a high amount over a normal cache. To help, an algorithm was developed based on expect cache behaviour for misses and communication to help size the caches of readers and writers.</p> <h2 id="conclusion">Conclusion</h2> <p>This is a snapshot of some concurrently readable datastructures, and how they are implemented and useful in your projects. Using them in <a href="https://github.com/kanidm/kanidm/blob/master/README.md">Kanidm</a> we have already seen excellent performance and scaling of the server, with very little effort for tuning. We plan to adapt these for use in 389 Directory Server too. Stay tuned!</p> Packaging and the Security Proposition Thu, 19 Dec 2019 00:00:00 +0000 Unknown https://fy.blackhats.net.au/blog/2019-12-19-packaging-and-the-security-proposition/ https://fy.blackhats.net.au/blog/2019-12-19-packaging-and-the-security-proposition/ <h1 id="packaging-and-the-security-proposition">Packaging and the Security Proposition</h1> <p>As a follow up to my post on distribution packaging, it was commented by Fraser Tweedale (@hackuador) that traditionally the &quot;security&quot; aspects of distribution packaging was a compelling reason to use distribution packages over &quot;upstreams&quot;. I want to dig into this further.</p> <h2 id="why-does-c-need-securing">Why does C need &quot;securing&quot;</h2> <p>C as a language is <em>unsafe</em> in every meaning of the word. The best C programmers on the planet are incapable of writing a secure program. This is because to code in C you have to express a concurrent problem, into a language that is linearised, which is compiled relying on undefined behaviour, to be executed on an asynchronous concurrent out of order CPU. What could possibly go wrong?!</p> <p>There is a lot you need to hold in mind to make C work. I can tell you now that I spend a majority of my development time thinking about the code to change rather than writing C because of this!</p> <p>This has led to C based applications having just about every security issue known to man.</p> <h2 id="how-is-c-secured">How is C &quot;secured&quot;</h2> <p>So as C is security swiss cheese, this means we have developed processes around the language to soften this issue - for example advice like patch and update continually as new changes are continually released to resolve issues.</p> <p>Distribution packages have always been the &quot;source&quot; of updates for these libraries and applications. These packages are maintained by humans who need to update these packages. This means when a C project releases a fix, these maintainers would apply the patch to various versions, and then release the updates. These library updates due to C's dynamic nature means when the machine is next rebooted (yes rebooted, not application restarted) that these fixes apply to all consumers who have linked to that library - change one, fix everything. Great!</p> <p>But there are some (glaring) weaknesses to this model. C historically has little to poor application testing so many of these patches and their effects can't be reproduced. Which also subsequently means that consuming applications also aren't re-tested adequately. It can also have impacts where a change to a shared library can impact a consuming application in a way that was unforseen as the library changed.</p> <h2 id="the-dirty-secret">The Dirty Secret</h2> <p>The dirty secret of many of these things is that &quot;thoughts and prayers&quot; is often the testing strategy of choice when patches are applied. It's only because humans carefully think about and write tiny amounts of C that we have any reliability in our applications. And we already established that it's nearly impossible for humans to write correct C ...</p> <h2 id="why-are-we-doing-this">Why Are We Doing This?</h2> <p>Because C linking and interfaces are so fragile, and due to the huge scope in which C can go wrong due to being a memory unsafe language, distributions and consumers have learnt to fear <em>version changes</em>. So instead we patch ancient C code stacks, barely test them, and hope that our castles of sand don't fall over, all so we can keep &quot;the same version&quot; of a program to avoid changing it as much as possible. Ironically this makes those stacks even worse because we've developed infinite numbers of bespoke barely tested packages that people rely on daily.</p> <p>To add more insult to this, most of this process is manual - humans monitor mailing lists, and have to know what code needs what patch, and when in what release streams. It's a monumental amount of human time and labour involved to keep the sand castles standing. This manual involvement is what leads to information overload, and maintainers potentially missing security updates or releases that causes many distribution packages to be outdated, missing patches, or vulnerable more often than not. In other cases packages continue to be shipped that are unmaintained or have no upstream, so any issues that may exist are unknown or unresolved.</p> <h2 id="distribution-security">Distribution Security</h2> <p>This means all of platform and distribution security comes to one factor.</p> <p><em>A lot of manual human labour.</em></p> <p>It's is only because distributions have so many volunteers or paid staff, that this entire system continues to progress to give the illusion of security and reliability. When it fails, it fails silently.</p> <p>Heartbleed really <a href="https://en.wikipedia.org/wiki/Heartbleed#Root_causes,_possible_lessons,_and_reactions">dragged the poor state of C security into the open</a> , and it's still not been addressed.</p> <p>When people say &quot;how can we secure docker/flatpak/Rust&quot; like we do with distributions, I say: &quot;Do we really secure distributions at all?&quot;. We only have a veneer of best effort masquerading as a secure supply chain.</p> <h2 id="a-different-model">A Different Model ...</h2> <p>So let's look briefly at Rust and how you package it today (against distribution maintainer advice).</p> <p>Because it's staticly linked, each application must be rebuilt if a library changes. Because the code comes from a central upstream, there are automated tools to find security issues (like cargo audit). The updates are pulled from the library as a whole working tested unit, and then built into our application to to recieve further testing and verification of the application as a whole singular functional unit.</p> <p>These dependencies once can then be vendored to a tar (allowing offline builds and some aspects of reproducability). This vendor.tar.gz is placed into the source rpm along with the application source, and then built.</p> <p>There is a much stronger pipeline of assurances here! And to further aid Rust's cause, because it is a memory <em>safe</em> language, it eliminates most of the security issues that C is afflicted by, causing security updates to be far fewer, and to often affect higher level or esoteric situations. If you don't believe me, look at the low frequency, and low severity <a href="https://github.com/RustSec/advisory-db/commits/master">commits for the rust advisory-db</a></p> <p>People have worried that because Rust is staticly linked we'll have to rebuild it and update it continually to keep it secure - I'd say because it's Rust we'll have stronger guarantees at build that security issues are less likely to exist and we won't have to ship updates nearly as often as a C stack.</p> <p>Another point to make is Rust libraries don't release patches - because of Rust's stronger guarantees at compile time and through integrated testing, people are less afraid of updates to versions. We are very unlikely to see Rust releasing patches, rather than just shipping &quot;updates&quot; to libraries and expecting you to update. Because these are staticly linked, we don't have to worry about versions for other libraries on the platform, we only need to assure the <em>application</em> is currently working as intended. Because of the strong typing those interfaces of those libraries has stronger compile time guarantees at build time, meaning the issues around shared object versioning and symbol/version mismatching simply don't exist - one of the key reasons people became version change averse in the first place.</p> <h2 id="so-why-not-package-all-the-things">So Why Not Package All The Things?</h2> <p>Many distribution packagers have been demanding a C-like model for Rust and others (<a href="../18/packaging_vendoring_and_how_it_s_changing.html">remember, square peg, round hole</a>). This means every single crate (library) is packaged, and then added to a set of buildrequires for the application. When a crate updates, it triggers the application to rebuild. When a security update for a library comes out, it rebuilds etc.</p> <p>This should sound familiar ... because it is. It's reinventing Cargo in a clean-room.</p> <p>RPM provides a way to manage dependencies. Cargo provides a way to manage dependencies.</p> <p>RPM provides a way to offline build sources. Cargo provides a way to offline build sources.</p> <p>RPM provides a way to patch sources. Cargo provides a way to update them inplace - and patch if needed.</p> <p>RPM provides a way to ... okay you get the point.</p> <p>There is also a list of what we won't get from distribution packages - remember distribution packages are the <a href="../18/packaging_vendoring_and_how_it_s_changing.html">C language packaging system</a></p> <p>We won't get the same level of attention to detail, innovation and support as the upstream language tooling has. Simply put, users of the language just won't use distribution packages (or toolchains, libraries ...) in their workflows.</p> <p>Distribution packages can't offer is the integration into tools like cargo-audit for scanning for security issues - that needs still needs Cargo, not RPM, meaning the RPM will need to emulate what Cargo does exactly.</p> <p>Using distribution packages means you have an untested pipeline that may add more risks now. Developers won't use distribution packages - they'll use cargo. Remember applications work best as they are tested and developed - outside of that environment they are an unknown.</p> <p>Finally, the distribution maintainers security proposition is to secure our <em>libraries</em> - for distributions only. That's acting in self interest. Cargo is offering a way to secure upstream so that everyone benefits. That means less effort and less manual labour all around. And secure libraries are not the full picture. Secure <em>applications</em> is what matters.</p> <p>The large concerning factor is the sheer amount of <em>human effort</em>. We would spend hundreds if not thousands of hours to reinvent a functional tool in a disengaged manner, just so that we can do things as they have always been done in C - for the benefit of distributions individually rather than languages upstream.</p> <h2 id="what-is-the-point">What is the Point</h2> <p>Again - as a platform our role is to provide <em>applications</em> that people can trust. The way we provide these applications is never going to be one size fits all. Our objective isn't to secure &quot;this library&quot; or &quot;that library&quot;, it's to secure <em>applications</em> as a functional whole. That means that companies shipping those applications, should hire maintainers to work on those applications to secure their stacks.</p> <p>Today I honestly think Rust has a better security and updating story than C packages ever has, powered by automation and upstream integration. Let's lean on that, contribute to it, and focus on shipping applications instead of reinventing tools. We need to accept our current model is focused on C, that developers have moved around distribution packaging, and that we need to change our approach to eliminate the large human risk factor that currently exists.</p> <p>We can't keep looking to the models of the past, we need to start to invest in new methods for the future.</p> <p>Today, distributions should focus on supporting and distributing <em>applications</em> and work with native language supply chains to enable this.</p> <p>Which is why I'll keep using cargo's tooling and auditing, and use distribution packages as a delievery mechanism for those applications.</p> <h2 id="what-could-it-look-like">What Could it Look Like?</h2> <p>We have a platform that updates as a whole (Fedora Atomic comes to mind ...) with known snapshots that are tested and well known. This platform has methods to run applications, and those applications are isolated from each other, have their own libraries, and security audits.</p> <p>And because there are now far fewer moving parts, quality is easier to assert, understand, and security updates are far easier and faster, less risky.</p> <p>It certainly sounds a lot like what macOS and iOS have been doing with a read-only base, and self-contained applications within that system.</p> Packaging, Vendoring, and How It's Changing Wed, 18 Dec 2019 00:00:00 +0000 Unknown https://fy.blackhats.net.au/blog/2019-12-18-packaging-vendoring-and-how-it-s-changing/ https://fy.blackhats.net.au/blog/2019-12-18-packaging-vendoring-and-how-it-s-changing/ <h1 id="packaging-vendoring-and-how-it-s-changing">Packaging, Vendoring, and How It's Changing</h1> <p>In today's thoughts, I was considering packaging for platforms like opensuse or other distributions and how that interacts with language based packaging tools. This is a complex and ... difficult topic, so I'll start with my summary:</p> <p>Today, distributions should focus on supporting and distributing <em>applications</em> and work with native language supply chains to enable this.</p> <h2 id="distribution-packaging">Distribution Packaging</h2> <p>Let's start by clarifying what distribution packaging is. This is your linux or platforms method of distributing it's programs libraries. For our discussion we really only care about linux so say suse or fedora here. How macOS or FreeBSD deal with this is quite different.</p> <p>Now these distribution packages are built to support certain workflows and end goals. Many open source C projects release their source code in varying states, perhaps also patches to improve or fix issues. These code are then put into packages, dependencies between them established due to dynamic linking, they are signed for verification purposes and then shipped.</p> <p>This process is really optimised for C applications. C has been the &quot;system language&quot; for many decades now, and so we can really see these features designed to promote - and fill in gaps - for these applications.</p> <p>For example, C applications are dynamically linked. Because of this it encourages package maintainers to &quot;split&quot; applications into smaller units that can have shared elements. An example that I know is openldap which may be a single source tree, but often is packaged to multiple parts such as the libldap.so, lmdb, openldap-client applications, it's server binary, and probably others. The package maintainer is used to taking their scalpels and carefully slicing sources into elegant packages that can minimise how many things are installed to what is &quot;just needed&quot;.</p> <p>We also see other behaviours where C shared objects have &quot;versions&quot;, which means you can install multiple versions of them at once and programs declare in their headers which library versions they want to consume. This means a distribution package can have many versions of the same thing installed!</p> <p>This in mind, the linking is simplistic and naive. If a shared object symbol doesn't exist, or you don't give it the &quot;right arguments&quot; via a weak-compile time contract, it's likely bad things (tm) will happen. So for this, distribution packaging provides the stronger assertions about &quot;this program requires that library version&quot;.</p> <p>As well, in the past the internet was a more ... wild place, where TLS wasn't really widely used. This meant that to gain strong assertions about the source of a package <em>and</em> that it had not been tampered, tools like GPG were used.</p> <h2 id="what-about-ruby-or-python">What about Ruby or Python?</h2> <p>Ruby and Python are very different languages compared to C though. They don't have information in their programs about what versions of software they require, and how they mesh together. Both languages are interpreted, and simply &quot;import library&quot; by name, searching a filesystem path for a library matching <em>regardless of it's version</em>. Python then just loads in that library as source straight to the running vm.</p> <p>It's already apparent how we'll run into issues here. What if we have a library &quot;foo&quot; that has a different function interface between version 1 and version 2? Python applications only request access to &quot;foo&quot;, not the version, so what happens if the wrong version is found? What if it's not found?</p> <p>Some features here are pretty useful from the &quot;distribution package&quot; though. Allowing these dynamic tools to have their dependencies requested from the &quot;package&quot;, and having the package integrity checked for them.</p> <p>But overtime, conflicts started, and issues arose. A real turning point was ruby in debian/ubuntu where debian package maintainers (who are used to C) brought out the scalpel and attempted to slice ruby down to &quot;parts&quot; that could be reused form a C mindset. This led to a combinations of packages that didn't make sense (rubygems minus TLS, but rubygems requires https), which really disrupted the communities.</p> <p>Another issue was these languages as they grew in popularity had different projects requiring <em>different</em> versions of libraries - which as before we mentioned isn't possible beside library search path manipulations which is frankly user hostile.</p> <p>These issues (and more) eventually caused these communities as a whole to <em>stop recommending</em> distribution packages.</p> <p>So put this history in context. We have Ruby (1995) and Python (1990), which both decided to avoid distribution packages with their own tools aka rubygems (2004) and pip (2011), as well as tools to manage multiple parallel environments (rvm, virtualenv) that were per-application.</p> <h2 id="new-kids-on-the-block">New kids on the block</h2> <p>Since then I would say three languages have risen to importance and learnt from the experiences of Ruby - This is Javascript (npm/node), Go and Rust.</p> <p>Rust went further than Ruby and Python and embedded distribution of libraries into it's build tools from an early date with Cargo. As Rust is staticly linked (libraries are build into the final binary, rather than being dynamicly loaded), this moves all dependency management to build time - which prevents runtime library conflict. And because Cargo is involved and controls all the paths, it can do things such as having multiple versions available in a single build for different components and coordinating all these elements.</p> <p>Now to hop back to npm/js. This ecosystem introduced a new concept - micro-dependencies. This happened because javascript doesn't have dead code elimination. So if given a large utility library, and you call one function out of 100, you still have to ship all 99 unused ones. This means they needed a way to manage and distribute hundreds, if not thousands of tiny libraries, each that did &quot;one thing&quot; so that you pulled in &quot;exactly&quot; the minimum required (that's not how it turned out ... but it was the intent).</p> <p>Rust also has inherited a similar culture - not to the same extreme as npm because Rust DOES have dead code elimination, but still enough that my concread library with 3 dependencies pulls in 32 dependencies, and kanidm from it's 30 dependencies, pulls in 365 into it's graph.</p> <p>But in a way this also doesn't matter - Rust enforces strong typing at compile time, so changes in libraries are detected before a release (not after like in C, or dynamic languages), and those versions at build are used in production due to the static linking.</p> <p>This has led to a great challenge is distribution packaging for Rust - there are so many &quot;libraries&quot; that to package them all would be a monumental piece of human effort, time, and work.</p> <p>But once again, we see the distribution maintainers, scalpel in hand, a shine in their eyes looking and thinking &quot;excellent, time to package 365 libraries ...&quot;. In the name of a &quot;supply chain&quot; and adding &quot;security&quot;.</p> <p>We have to ask though, is there really <em>value</em> of spending all this time to package 365 libraries when Rust functions so differently?</p> <h2 id="what-are-you-getting-at-here">What are you getting at here?</h2> <p>To put it clearly - distribution packaging isn't a &quot;higher&quot; form of distributing software. Distribution packages are not the one-true solution to distribute software. It doesn't magically enable &quot;security&quot;. Distribution Packaging <em>is</em> the C language source and binary distribution mechanism - and for that it works great!</p> <p>Now that we can frame it like this we can see <em>why</em> there are so many challenges when we attempt to package Rust, Python or friends in rpms.</p> <p>Rust isn't C. We can't think about Rust like C. We can't secure Rust like C.</p> <p>Python isn't C. We can't think about Python like C. We can't secure Python like C.</p> <p>These languages all have their own quirks, behaviours, flaws, benefits, and goals. They need to be distributed in unique ways appropriate to those languages.</p> <h2 id="an-example-of-the-mismatch">An example of the mismatch</h2> <p>To help drive this home, I want to bring up FreeIPA. FreeIPA has a lot of challenges in packaging due to it's <em>huge</em> number of C, Python and Java dependencies. Recently on twitter it was annouced that &quot;FreeIPA has been packaged for debian&quot; as the last barrier (being dogtag/java) was overcome to package the hundreds of required dependencies.</p> <p>The inevitable outcome of debian now packaging FreeIPA will be:</p> <ul> <li>FreeIPA will break in some future event as one of the python or java libraries was changed in a way that was not expected by the developers or package maintainers.</li> <li>Other applications may be &quot;held back&quot; from updating at risk/fear of breaking FreeIPA which stifles innovation in the java/python ecosystems surrounding.</li> </ul> <p>It won't be the fault of FreeIPA. It won't be the fault of the debian maintainers. It will be that we are shoving square applications through round C shaped holes and hoping it works.</p> <h2 id="so-what-does-matter">So what does matter?</h2> <p>It doesn't matter if it's Kanidm, FreeIPA, or 389-ds. End users want to consume <em>applications</em>. How that application is developed, built and distributed is a secondary concern, and many people will go their whole lives never knowing how this process works.</p> <p>We need to stop focusing on packaging <em>libraries</em> and start to focus on how we distribute <em>applications</em>.</p> <p>This is why projects like docker and flatpak have surprised traditional packaging advocates. These tools are about how we ship <em>applications</em>, and their build and supply chains are separated from these.</p> <p>This is why I have really started to advocate and say:</p> <p>Today, distributions should focus on supporting and distributing <em>applications</em> and work with native language supply chains to enable this.</p> <p>Only we accept this shift, we can start to find value in distributions again as sources of trusted applications, and how we see the distribution as an application platform rather than a collection of tiny libraries.</p> <p>The risk of not doing this is alienating communities (again) from being involved in our platforms.</p> <h2 id="follow-up">Follow Up</h2> <p>There have been some major comments since:</p> <p>First, there is now a C package manager named <a href="https://conan.io/">conan</a> . I have no experience with this tool, so at a distance I can only assume it works well for what it does. However it was noted it's not gained much popularity, likely due to the fact that distro packages are the current C language distribution channels.</p> <p>The second was about the security components of distribution packaging and this - that topic is so long I've written <a href="../19/packaging_and_the_security_proposition.html">another post</a> about the topic instead, to try and keep this post focused on the topic.</p> <p>Finally, The Fedora Modularity effort is trying to deal with some of these issues - that modules, aka applications have different cadences and requirements, and those modules can move more freely from the base OS.</p> <p>Some of the challenges have been <a href="https://lwn.net/Articles/805180/">explored by LWN</a> and it's worth reading. But I think the underlying issue is that again we are approaching things in a way that may not align with reality - people are looking at modules as libraries, not applications which is causing things to go sideways. And when those modules are installed, they aren't <a href="https://lwn.net/ml/fedora-devel/4040208.GuUW4P68lE@marvin.jharris.pw/">isolated from each other</a> , meaning we are back to square one, with a system designed only for C. People are <a href="https://lwn.net/ml/fedora-devel/CAB-QmhSdQ+Gwuf6gnLaiziMY+nxNgTSFaUyzRuJqO6sN7_6wzw@mail.gmail.com/">starting to see that</a> but the key point is continually missed - that modularity should be about <em>applications</em> and their isolation not about <em>multiple library versions</em>.</p> Fixing opensuse virtual machines with resume Sun, 15 Dec 2019 00:00:00 +0000 Unknown https://fy.blackhats.net.au/blog/2019-12-15-fixing-opensuse-virtual-machines-with-resume/ https://fy.blackhats.net.au/blog/2019-12-15-fixing-opensuse-virtual-machines-with-resume/ <h1 id="fixing-opensuse-virtual-machines-with-resume">Fixing opensuse virtual machines with resume</h1> <p>Today I hit an unexpected issue - after changing a virtual machines root disk to scsi, I was unable to boot the machine.</p> <p>The host is opensuse leap 15.1, and the vm is the same. What's happening!</p> <p>The first issue appears to be that opensuse 15.1 doesn't support scsi disks from libvirt. I'm honestly not sure what's wrong here.</p> <p>The second is that by default opensuse leap configures suspend and resume to disk - by it uses the pci path instead of a swap volume UUID. So when you change the bus type, it renames the path making the volume inaccessible. This causes boot to fail.</p> <p>To work around you can remove &quot;resume=/disk/path&quot; from the cli. Then to fix it permanently you need:</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>transactional-update shell </span><span>vim /etc/default/grub </span><span># Edit this line to remove &quot;resume&quot; </span><span>GRUB_CMDLINE_LINUX_DEFAULT=&quot;console=ttyS0,115200 resume=/dev/disk/by-path/pci-0000:00:07.0-part3 splash=silent quiet showopts&quot; </span><span> </span><span>vim /etc/default/grub_installdevice </span><span># Edit the path to the correct swap location as by ls -al /dev/disk/by-path </span><span>/dev/disk/by-path/pci-0000:00:07.0-part3 </span></code></pre> <p>I have reported these issues, and I hope they are resolved.</p> Password Quality and Badlisting in Kanidm Sat, 07 Dec 2019 00:00:00 +0000 Unknown https://fy.blackhats.net.au/blog/2019-12-07-password-quality-and-badlisting-in-kanidm/ https://fy.blackhats.net.au/blog/2019-12-07-password-quality-and-badlisting-in-kanidm/ <h1 id="password-quality-and-badlisting-in-kanidm">Password Quality and Badlisting in Kanidm</h1> <p>Passwords are still a required part of any IDM system. As much as I wish for Kanidm to only support webauthn and stronger authentication types, at the end of the day devices can be lost, destroyed, some people may not be able to afford them, some clients aren't compatible with them and more.</p> <p>This means the current state of the art is still multi-factor auth. Something you have and something you know.</p> <p>Despite the presence of the multiple factors, it's still important to quality check passwords. Microsoft's Azure security team <a href="https://techcommunity.microsoft.com/t5/Azure-Active-Directory-Identity/Your-Pa-word-doesn-t-matter/ba-p/731984">have written about passwords</a>, and it really drives home the current situation. I would certainly trust these people at Microsoft to know what they are talking about given the scale of what they have to defend daily.</p> <p>The most important take away is that trying to obscure the password from a bruteforce is a pointless exercise because passwords end up in password dumps, they get phished, keylogged, and more. MFA matters!</p> <p>It's important here to look at the &quot;easily guessed&quot; and &quot;credential stuffing&quot; category. That's what we really want to defend against with password quality, and MFA protects us against keylogging, phising (only webauthn), and reuse.</p> <h2 id="can-we-avoid-this">Can we Avoid This?</h2> <p>Yes! Kanidm supports a &quot;generated&quot; password that is a long, high entropy password that should be stored in a password manager or similar tool to prevent human knowledge. This fits to our &quot;device as authentication&quot; philosophy. You authenticate to your device (phone, laptop etc), and then the devices stored passwords authenticate you from that point on. This has the benefit that devices and password managers generally perform better checking of the target where we enter the password, making phising less likely.</p> <p>But sometimes we can't rely on this, so we still need human-known passwords, so we still take steps to handle these.</p> <h2 id="quality">Quality</h2> <p>In the darkages, we would decree that human known passwords had to have a number, symbol, roman numeral, no double letters, one capital letter, two kanji and no symbols that could be part of an sql injection because of that one legacy system we can't patch.</p> <p>This lead to people making horrid, un-rememberable passwords in leetspeak, or giving up altogether and making the excellent &quot;Password1&quot; (which passes AD minimum password requirements on server 2003).</p> <p>What we really want is entropy, length, and memorability. Dropbox made a great library for this called zxcvbn, which has since <a href="https://docs.rs/zxcvbn/2.0.0/zxcvbn/index.html">been ported to rust</a> . I highly recommend it.</p> <p>This library is great because it focuses on entropy, and then if the password doesn't meet these requirements, the library recommends ways to improve. This is excellent for human interaction and experience, guiding people to create better passwords that they can remember, rather than our outdated advice of the complex passwords as described above.</p> <h2 id="badlisting">Badlisting</h2> <p>Badlisting is another great technique for improving password quality. It's essentially a blocklist of passwords that people are not allowed to set. This way you can have corporate-specific breach lists, or the top 10k most used passwords badlisted to prevent users using them. For example, &quot;correct horse battery staple&quot; may be a strong password, but it's well known thanks to xkcd.</p> <p>It's also good for preventing password reuse in your company if you are phished and the credentials privately notified to you as some of the regional CERT's do, allowing you to block these without them being in a public breach list.</p> <p>This is important as many bots will attempt to spam these passwords against accounts (rate limiting auth and soft-locking accounts also helps to delay these attack styles).</p> <h2 id="in-kanidm">In Kanidm</h2> <p>In Kanidm, we chose to use both approaches. First we check the password with zxcvbn, then we ensure it's not in a badlist.</p> <p>In order to minimise the size of the badlist, the badlist uses case insensitive storage so that multiple variants of &quot;password&quot; and &quot;PasSWOrD&quot; are only listed once. We also preprocessed the badlist with zxcvbn to remove any passwords that it would have denied from being entered. The preprocessor tool will be shipped with kanidm so that administrators can preprocess their own lists before adding them to the badlist configuration.</p> <h2 id="creating-a-badlist">Creating a Badlist</h2> <p>I decided to do some analysis on a well known set of passwords maintained on the <a href="https://github.com/danielmiessler/SecLists/tree/master/Passwords">seclists repository</a> . Apparently this is what pentesters reach for when they want to bruteforce or credential stuff on a domain.</p> <p>I analysed this in three ways. The first was as a full set of passwords (about 25 million) and a smaller but &quot;popular&quot; set in the &quot;rockyou&quot; files which is about 60,000 passwords. Finally I did an analysis of the rockyou + top 10 million most common (which combined was 1011327 unique passwords, so about 50k of the rockyou set is from top 10 million).</p> <p>From the 25 million set I ran this through a preprocessor tool that I developed for kanidm. It eliminated anything less than a score of 3 and no length rule. This showed that zxcvbn was able to prevent 80% of these inputs from being allowed. If I was to ship this full list, this would contain 4.8 million badlisted passwords. It's pretty amazing already that zxcvbn stops 80% of bad passwords that end up in breaches from being able to be used, with the remaining 20% likely to be complex passwords that just got dumped by other means.</p> <p>However, for the badlist in Kanidm, I decided to start with &quot;what's popular&quot; for now, and to allow sites to add extra content if they desire. This meant that I focused instead on the &quot;rockyou&quot; password set instead.</p> <p>From the rockyou set I did more tests. zxcvbn has a concept of scores, and we can have policy to request a minimum score is present to allow the password. I did a score 3 test, a score 3 with min pw len 10 and a score 4 test. This showed the following results which has the % blocked by zxcvbn and the no. that passed which will required badlisting as zxcvbn can't detect them (today).</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>TEST | % blocked | no. passed </span><span>--------------------------------- </span><span> s3 | 98.3% | 1004 </span><span> s3 + 10 | 98.9% | 637 </span><span> s4 | 99.7% | 133 </span></code></pre> <p>Personally, it's quite hilarious that &quot;2fast2furious&quot; passed the score 3 check, and &quot;30secondstomars&quot; and &quot;dracomalfoy&quot; passed the score 4 check, but who am I to judge - that's what bad lists are for.</p> <p>More seriously, I found it interesting to see the effect of the check on length - not only was the preprocessor step faster, but alone that eliminated ~400 passwords that would have &quot;passed&quot; on score 3.</p> <p>Finally, from the rockyou + 10m set, the results show the following in the same conditions.</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>TEST | % blocked | no. passed </span><span>--------------------------------- </span><span> s3 | 89.9% | 101349 </span><span> s3 + 10 | 92.4% | 76425 </span><span> s4 | 96.5% | 34696 </span></code></pre> <p>This shows that a very &quot;easy&quot; win is to enforce password length, in addition to entropy checkers like zxcvbn, which are effective to block 92% of the most common passwords in use on a broad set and 98% of what a pentester will look for (assuming rockyou lists). If you have a high security environment you should consider setting zxcvbn to request passwords of score 4 (the maximum), given that on the 10m set it had a 96.5% block rate.</p> <h2 id="conclusions">Conclusions</h2> <p>You should use zxcvbn, it's a great library, which quickly reduces a huge amount of risk from low quality passwords.</p> <p>After that your next two strongest controls are password length, and being able to support badlisting.</p> <p>Better yet, use MFA like Webauthn as well, and support server-side generated high-entropy passwords!</p> Rust 2020 - helping to get rust deployed Thu, 28 Nov 2019 00:00:00 +0000 Unknown https://fy.blackhats.net.au/blog/2019-11-28-rust-2020-helping-to-get-rust-deployed/ https://fy.blackhats.net.au/blog/2019-11-28-rust-2020-helping-to-get-rust-deployed/ <h1 id="rust-2020-helping-to-get-rust-deployed">Rust 2020 - helping to get rust deployed</h1> <p>This is my contribution to Rust 2020, where community members put forward ideas on what they thing Rust should aim to achieve in 2020.</p> <p>In my view, Rust has had an amazing adoption by developers, and is great if you are in a position to deploy it in your own infrastructure, but we have yet to really see Rust make it to broad low-level components (IE in a linux distro or other infrastructure).</p> <p>As someone who works on &quot;enterprise&quot; software (389-ds) and my own IDM project (kanidm), there is a need to have software packaged and distributed. We can not ask our consumers to build and compile these tools. One could view it as a chain, where I develop software in a language, it's packaged for a company (like SUSE), and then consumed by a customer (could be anyone!) who provides a service to others (indirect users).</p> <p>Rust however has always been modeled that there is no &quot;middle&quot; section. You have either a developer who's intent is to develop for other developers. This is where Rust ideas like crates.io becomes involved. Alternately, you have a larger example in firefox, where developers build a project and can &quot;bundle&quot; everything into a whole unit that is then distributed directly to customers.</p> <p>The major difference is that in the intermediate distribution case, we have to take on different responsibilities such as security auditing, building, ensuring dependencies exist etc.</p> <p>So this leads me to:</p> <h2 id="1-cargo-vendor-needs-some-love">1: Cargo Vendor Needs Some Love</h2> <p>Cargo vendor today is quite confusing in some scenarios, and it's not clear how to have it work for projects that require offline builds. I have raised issues about this, but largely they have been un-acted upon.</p> <h2 id="2-cargo-is-difficult-to-use-in-mixed-language-projects">2: Cargo is Difficult to Use in Mixed Language Projects</h2> <p>A large value proposition of Rust is the ability to use it with FFI and C. This is great if you say have cargo compile your C code for you.</p> <p>But most major existing projects don't. They use autotools, cmake, or maybe even meson or more esoteric, waf (looking at you samba). Cargo's extreme opinionation in this area makes it extremely difficult to integrate Rust into an existing build system reliably. It's hard to point to one fault, as much as a broader &quot;lack of concern&quot; in the space, and I think cargo needs to evolve improvements to help allow Rust to be used from other build tools.</p> <h2 id="3-rust-moves-too-fast">3: Rust Moves Too Fast</h2> <p>A lot of &quot;slower&quot; enterprise companies want to move slowly, including compiler versions. Admittedly, this conservative behaviour is because of the historical instability of gcc versions and how it can change or affect your code between releases. Rust doesn't suffer this, but people are still wary of fast version changes. This means Rustc + Cargo will get pinned to some version that may be 6 months old.</p> <p>However crate authors don't consider this - they will use the latest and greatest features from stable (and sometimes still nightly ... grrr) in releases. Multiple times I have found that on my development environment even with a 3 month old compiler, dependencies won't build.</p> <p>Compounding this, crates.io doesn't distinguish a security release from a feature one. Crates also encourages continuall version bumping, rather than maintenence of versioned branches. IE version 0.4.3 of a crate with a security fix will become 0.4.4, but then a feature update to include try_from may go to 0.4.5 as it &quot;adds&quot; to the api, or they use it internally as a cleanup.</p> <p>Part of this issue is that Rust applications need to be treated closer to docker, as static whole units where only the resulting binary is supported rather than the toolchain that built it. But that only works on pure Rust applications - any mixed C + Rust application will hit this issue due to the difference between a system Rust version and what crate dependencies publish.</p> <p>So I think that from this it leads to:</p> <h2 id="3-1-crates-need-to-indicate-a-minimum-supported-compiler-version">3.1: Crates need to indicate a minimum supported compiler version</h2> <p>Rust has &quot;toyed&quot; with the idea of editions, but within 2018 we've seen new features like maybeuninit and try_from land, which within an &quot;edition&quot; caused crates to stop worked on older compilers.</p> <p>As a result, editions I think is &quot;too broad&quot; and people will fear incrementing it, and Rust will add features without changing edition anyway. Instead Rust needs to consider following up on the minimum supported rust version flag RFC. Rust has made it pretty clear the only &quot;edition&quot; flag that matters is the rust compiler version based on crate developers and what they are releasing.</p> <h2 id="3-2-rust-needs-to-think-what-s-our-end-goal">3.2: Rust Needs to Think &quot;What's Our End Goal&quot;</h2> <p>Rust is still moving incredibly fast, and I think in a way we need to ask ... when will Rust be finished? When will it as a language go from continually rapid growth to stable and known feature sets? The idea of Rust editions acts as though this has happened (saying we change only every few years) when this is clearly not the case. Rust is evolving release-on-release, every 6 weeks.</p> <h2 id="4-zero-cost-needs-to-factor-in-human-cost">4: Zero Cost Needs to Factor in Human Cost</h2> <p>My final wish for Rust is that sometimes we are so obsessed with the technical desire for zero cost abstraction, that we forget the high human cost and barriers that can exist as a result making feature adoption challenging. Rust has had a great community that treats people very well, and I think sometimes we need to extend that into feature development, to really consider the human cognitive cost of a feature.</p> <p>Summarised - what's the benefit of a zero cost abstraction if people can not work out how to use it?</p> <h2 id="summary">Summary</h2> <p>I want to see Rust become a major part of operating systems and how we build computer systems, but I think that we need to pace ourselves, improve our tooling, and have some better ideas around what Rust should look like.</p> Recovering LVM when a device is missing with a cache pool lv Tue, 26 Nov 2019 00:00:00 +0000 Unknown https://fy.blackhats.net.au/blog/2019-11-26-recovering-lvm-when-a-device-is-missing-with-a-cache-pool-lv/ https://fy.blackhats.net.au/blog/2019-11-26-recovering-lvm-when-a-device-is-missing-with-a-cache-pool-lv/ <h1 id="recovering-lvm-when-a-device-is-missing-with-a-cache-pool-lv">Recovering LVM when a device is missing with a cache pool lv</h1> <p>I had a heartstopping moment today: my after running a command lvm proudly annouced it had removed an 8TB volume containing all of my virtual machine backing stores.</p> <h2 id="everyone-a-short-view-back-to-the-past">Everyone, A short view back to the past ...</h2> <p>I have a home server, with the configured storage array of:</p> <ul> <li>2x 8TB SMR (Shingled Magnetic Recording) archive disks (backup target)</li> <li>2x 8TB disks (vm backing store)</li> <li>2x 1TB nvme SSD (os + cache)</li> </ul> <p>The vm backing store also had a lvm cache segment via the nvme ssds in a raid 1 configuration. This means that the 2x8TB drives are in raid 1, and a partition on each of the nvme devices are in raid 1, then they are composed to allow the nvme to cache blocks from/to the 8TB array.</p> <p>Two weeks ago I noticed one of the nvme drives was producing IO errors indicating a fault of the device. Not wanting to risk corruption or other issues from growing out of hand, I immediately shutdown the machine and identified the nvme disk with the error.</p> <p>At this stage I took the precaution of imaging (dd) both the good and bad nvme devices to the archive array. Subsequently I completed a secure erase of the faulty nvme drive before returning it to the vendor for RMA.</p> <p>I then left the server offline as I was away from my home for more than a week and would not need, and was unable to monitor if the other drives would produce further errors.</p> <h2 id="returning-home">Returning home ...</h2> <p>I decided to ignore William of the past (always a bad idea) and to &quot;break&quot; the raid on the remaining nvme device so that my server could operate allowing me some options for work related tasks.</p> <p>This is an annoying process in lvm - you need to remove the missing device from the volume group as well as indicating to the array that it should no longer be in a raid state. This vgreduce is only for removing missing PV's, it shouldn't be doing anything else.</p> <p>I initiated the raid break process on the home, root and swap devices. The steps are:</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>vgreduce --removemissing &lt;vgname&gt; </span><span>lvconvert -m0 &lt;vgname&gt;/&lt;lvname&gt; </span></code></pre> <p>This occured without fault due to being present on an isolated &quot;system&quot; volume group, so the partial lvs were untouched and left on the remaining pv in the vg.</p> <p>When I then initiated this process on the &quot;data&quot; vg which contained the libvirt backing store, vgreduce gave me the terrifying message:</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>Removing logical volume &quot;libvirt_t2&quot;. </span></code></pre> <p>Oh no ~</p> <h2 id="recovery-attempts">Recovery Attempts</h2> <p>When a logical volume is removed, it can be recovered as lvm stores backups of the LVM metadata state in /etc/lvm/archive.</p> <p>My initial first reaction was that I was on a live disk, so I needed to backup this content else it would be lost on reboot. I chose to put this on the unaffected, and healthy SMR archive array.</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>mount /dev/mapper/archive-backup /archive </span><span>cp -a /etc/lvm /archive/lvm-backup </span></code></pre> <p>At this point I knew that randomly attempting commands would cause <em>further damage</em> and likely <em>prevent any ability to recover</em>.</p> <p>The first step was to activate ssh so that I could work from my laptop - rather than the tty with keyboard and monitor on my floor. It also means you can copy paste, which reduces errors. Remember, I'm booted on a live usb, which is why I reset the password.</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span># Only needed in a live usb. </span><span>passwd </span><span>systemctl start sshd </span></code></pre> <p>I then formulated a plan and wrote it out. This helps to ensure that I've thought through the recovery process and the risks, it helps be to be fully aware of the situation.</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>vim recovery-plan.txt </span></code></pre> <p>Into this I laid out the commands I would follow. Here is the plan:</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>bytes 808934440960 </span><span>data_00001-2096569583 </span><span> </span><span>dd if=/dev/zero of=/mnt/lv_temp bs=4096 count=197493760 </span><span>losetup /dev/loop10 /mnt/lv_temp </span><span>pvcreate --restorefile /etc/lvm/archive/data_00001-2096569583.vg --uuid iC4G41-PSFt-6vqp-GC0y-oN6T-NHnk-ivssmg /dev/loop10 </span><span>vgcfgrestore data --test --file /etc/lvm/archive/data_00001-2096569583.vg </span></code></pre> <p>Now to explain this: The situation we are in is:</p> <ul> <li>We have a removed data/libvirt_t2 logical volume</li> <li>The VG data is missing a single PV (nvme0). It still has three PVs (nvme1, sda1, sdb1).</li> <li>We can not restore metadata unless all devices are present as per the vgcfgrestore man page.</li> </ul> <p>This means, we need to make a replacement device to replace into the array, and then to restore the metadata with that.</p> <p>The &quot;bytes&quot; section you see, is the <em>size</em> of the missing nvme0 partition that was a member of this array - we need to create a loopback device of the same or greater size to allow us to restore the metadata. (dd, losetup)</p> <p>Once the loopback is created, we can then recreate the pv on the loopback device with the same UUID as the missing device.</p> <p>Once this is present, we can now restore the metadata as documented which should contain the logical volume.</p> <p>I ran these steps and it was all great until vgcfgrestore. I can not remember the exact error but it was along the lines of:</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>Unable to restore metadata as PV was missing for VG when last modification was performed. </span></code></pre> <p>Yep, the vgreduce command has changed the VG state, triggering a metadata backup, but because a device was missing at the time, we can not restore this metadata.</p> <h2 id="options">Options ...</h2> <p>At this point I had to consider alternate options. I conducted research into this topic as well to see if others had encountered this case (no one has ever not been able to restore their metadata apparently in this case ...). The options that I arrived at:</p> <ul> <li> <ol> <li>Restore the metadata from the nvme /root as it has older (but known) states - however I had recently expanded the libvirt_t2 volume from a live disk, meaning it may not have the correct part sizes.</li> </ol> </li> <li> <ol start="2"> <li>Attempt to extract the xfs filesystem with DD from the disk to begin a data recovery.</li> </ol> </li> <li> <ol start="3"> <li>Cry in a corner</li> </ol> </li> <li> <ol start="4"> <li>Use lvcreate with the &quot;same paramaters&quot; and hope that it aligns the start at the same location as the former data/libvirt_t2 allowing the xfs filesystem to be accessed.</li> </ol> </li> </ul> <p>All of these weren't good - especially not 3.</p> <p>I opted to attempt solution 1, and then if that failed, I would disconnect one of the 8TB disks, attempt solution 4, then if that ALSO failed, I would then attempt 2, finally all else lost I would begin solution 3. The major risk of relying on 4 and 2 is that LVM has dynamic geometry on disk, it does not always allocate contiguously. This means that attempting 4 with lvcreate may not create with the same geometry, and it may write to incorrect locations causing dataloss. The risk of 2 was again, due to the dynamic geometry what we recover may be re-arranged and corrupt.</p> <p>This mean option 1 was the best way to proceed.</p> <p>I mounted the /root volume of the host and using the lvm archive I was able to now restore the metadata.</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>vgcfgrestore data --test --file /system/etc/lvm/archive/data_00004-xxxx.vg </span></code></pre> <p>Once completed I performed an lvscan to refresh what block devices were available. I was then shown that every member of the VG data had conflicting seqno, and that the metadata was corrupt and unable to proceed.</p> <p>Somehow we'd made it worse :(</p> <h2 id="successful-procedure">Successful Procedure</h2> <p>At this point, faced with 3 options that were all terrible, I started to do more research. I finally discovered a post describing that the lvm metadata is stored on disk in the same format as the .vg files in the archive, and it's a ring buffer. We may be able to restore from these.</p> <p>To do so, you must <em>dd</em> out of the disk into a file, and then manipulate the file to only contain a single metadata entry.</p> <p>Remember how I made images of my disks before I sent them back? This was their time to shine.</p> <p>I did do a recovery plan with these commands too, but it was more evolving due to the paramaters involved so it changed frequently with the offsets. The plan was very similar to above - use a loop device as a stand in for the missing block device, restore the metadata, and then go from there.</p> <p>We know that LVM metadata occurs in the first section of the disk, just after the partition start. So to work out where this is we use gdisk to show the partitions in the backup image.</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span># gdisk /mnt/mion.nvme0n1.img </span><span>GPT fdisk (gdisk) version 1.0.4 </span><span>... </span><span> </span><span>Command (? for help): p </span><span>Disk /mnt/mion.nvme0n1.img: 2000409264 sectors, 953.9 GiB </span><span>Sector size (logical): 512 bytes </span><span>... </span><span> </span><span>Number Start (sector) End (sector) Size Code Name </span><span> 1 2048 1026047 500.0 MiB EF00 </span><span> 2 1026048 420456447 200.0 GiB 8E00 </span><span> 3 420456448 2000409230 753.4 GiB 8E00 </span></code></pre> <p>It's important to note the sector size flag, as well as the fact the output is in sectors.</p> <p>The LVM header occupies 255 sectors after the start of the partition. So this in mind we can now create a dd command to extract the needed information.</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>dd if=/mnt/mion.nvme0n1.img of=/tmp/lvmmeta bs=512 count=255 skip=420456448 </span></code></pre> <p>bs sets the sector size to 512, count will read from the start up to 255 sectors of size 'bs', and skip says to start reading after 'skip' * 'sector'.</p> <p>At this point, we can now copy this and edit the file:</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>cp /tmp/lvmmeta /archive/lvm.meta.edit </span></code></pre> <p>Within this file you can see the ring buffer of lvm metadata. You need to find the highest seqno that is a <em>complete record</em>. For example, my seqno = 20 was partial (is the lvm meta longer than 255, please contact me if you know!), but seqno=19 was complete.</p> <p>Here is the region:</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span># ^ more data above. </span><span>} </span><span># Generated by LVM2 version 2.02.180(2) (2018-07-19): Mon Nov 11 18:05:45 2019 </span><span> </span><span>contents = &quot;Text Format Volume Group&quot; </span><span>version = 1 </span><span> </span><span>description = &quot;&quot; </span><span> </span><span>creation_host = &quot;linux-p21s&quot; # Linux linux-p21s 4.12.14-lp151.28.25-default #1 SMP Wed Oct 30 08:39:59 UTC 2019 (54d7657) x86_64 </span><span>creation_time = 1573459545 # Mon Nov 11 18:05:45 2019 </span><span> </span><span>^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@data { </span><span>id = &quot;4t86tq-3DEW-VATS-1Q5x-nLLy-41pR-zEWwnr&quot; </span><span>seqno = 19 </span><span>format = &quot;lvm2&quot; </span></code></pre> <p>So from there you remove everything above &quot;contents = ...&quot;, and clean up the vgname header. It should look something like this.</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>contents = &quot;Text Format Volume Group&quot; </span><span>version = 1 </span><span> </span><span>description = &quot;&quot; </span><span> </span><span>creation_host = &quot;linux-p21s&quot; # Linux linux-p21s 4.12.14-lp151.28.25-default #1 SMP Wed Oct 30 08:39:59 UTC 2019 (54d7657) x86_64 </span><span>creation_time = 1573459545 # Mon Nov 11 18:05:45 2019 </span><span> </span><span>data { </span><span>id = &quot;4t86tq-3DEW-VATS-1Q5x-nLLy-41pR-zEWwnr&quot; </span><span>seqno = 19 </span><span>format = &quot;lvm2&quot; </span></code></pre> <p>Similar, you need to then find the bottom of the segment (look for the next highest seqno) and remove everything <em>below</em> the line: &quot;# Generated by LVM2 ...&quot;</p> <p>Now, you can import this metadata to the loop device for the missing device. Note I had to wipe the former lvm meta segment due to the previous corruption, which caused pvcreate to refuse to touch the device.</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>dd if=/dev/zero of=/dev/loop10 bs=512 count=255 </span><span>pvcreate --restorefile lvmmeta.orig.nvme1.edited --uuid iC4G41-PSFt-6vqp-GC0y-oN6T-NHnk-ivssmg /dev/loop10 </span></code></pre> <p>Now you can do a dry run:</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>vgcfgrestore --test -f lvmmeta.orig.nvme1.edited data </span></code></pre> <p>And the real thing:</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>vgcfgrestore -f lvmmeta.orig.nvme1.edited data </span><span>lvscan </span></code></pre> <p>Hooray! We have volumes! Let's check them, and ensure their filesystems are sane:</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>lvs </span><span>lvchange -ay data/libvirt_t2 </span><span>xfs_repair -n /dev/mapper/data-libvirt_t2 </span></code></pre> <p>If xfs_repair says no errors, then go ahead and mount!</p> <p>At this point, lvm started to resync the raid, so I'll leave that to complete before I take any further action to detach the loopback device.</p> <h2 id="how-to-handle-this-next-time">How to Handle This Next Time</h2> <p>The cause of this issue really comes from vgreduce --removemissing removing the device when a cache member can't be found. I plan to report this as a bug.</p> <p>However another key challenge was the inability to restore the lvm metadata when the metadata archive reported a missing device. This is what stopped me from being able to restore the array in the first place, even though I had a &quot;fake&quot; replacement. This is also an issue I intend to raise.</p> <p>Next time I would:</p> <ul> <li>Activate the array as a partial</li> <li>Remove the cache device first</li> <li>Then stop the raid</li> <li>Then perform the vgreduction</li> </ul> <p>I really hope this doesn't happen to you!</p> Upgrading OpenSUSE 15.0 to 15.1 Wed, 25 Sep 2019 00:00:00 +0000 Unknown https://fy.blackhats.net.au/blog/2019-09-25-upgrading-opensuse-15-0-to-15-1/ https://fy.blackhats.net.au/blog/2019-09-25-upgrading-opensuse-15-0-to-15-1/ <h1 id="upgrading-opensuse-15-0-to-15-1">Upgrading OpenSUSE 15.0 to 15.1</h1> <p>It's a little bit un-obvious how to do this. You have to edit the repo files to change the release version, then refresh + update.</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>sed -ri &#39;s/15\.0/15.1/&#39; /etc/zypp/repos.d/*.repo </span><span>zypper ref </span><span>zypper dup </span><span>reboot </span></code></pre> <p>Note this works on a transactional host too:</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>sed -ri &#39;s/15\.0/15.1/&#39; /etc/zypp/repos.d/*.repo </span><span>transactional-update dup </span><span>reboot </span></code></pre> <p>It would be nice if these was an upgrade tool that would attempt the upgrade and revert the repo files, or use temporary repo files for the upgrade though. It would be a bit nicer as a user experience than sed of the repo files.</p> Announcing Kanidm - A new IDM project Wed, 18 Sep 2019 00:00:00 +0000 Unknown https://fy.blackhats.net.au/blog/2019-09-18-announcing-kanidm-a-new-idm-project/ https://fy.blackhats.net.au/blog/2019-09-18-announcing-kanidm-a-new-idm-project/ <h1 id="announcing-kanidm-a-new-idm-project">Announcing Kanidm - A new IDM project</h1> <p>Today I'm starting to talk about my new project - Kanidm. Kanidm is an IDM project designed to be correct, simple and scalable. As an IDM project we should be able to store the identities and groups of people, authenticate them securely to various other infrastructure components and services, and much more.</p> <p>You can find the source for <a href="https://github.com/Firstyear/kanidm/blob/master/README.md">kanidm on github</a>.</p> <p>For more details about what the project is planning to achieve, and what we have already implemented please see the github.</p> <h2 id="what-about-389-directory-server">What about 389 Directory Server</h2> <p>I'm still part of the project, and working hard on making it the best LDAP server possible. Kanidm and 389-ds have different goals. 389 Directory Server is a globally scalable, distributed database, that can store huge amounts of data and process thousands of operations per second. 389-ds let's you build a system ontop, in any way you want. If you want an authentication system today, use 389-ds. We are even working on a self-service web portal soon too (one of our most requested features!). Besides my self, no one on the (amazing) 389 DS team has any association with kanidm (yet?).</p> <p>Kanidm is an opinionated IDM system, and has strong ideas about how authentication and users should be processed. We aim to be scalable, but that's a long road ahead. We also want to have more web integrations, client tools and more. We'll eventually write a kanidm to 389-ds sync tool.</p> <h2 id="why-not-integrate-something-with-389-why-something-new">Why not integrate something with 389? Why something new?</h2> <p>There are a lot of limitations with LDAP when it comes to modern web-focused auth processes such as webauthn. Because of this, I wanted to make something that didn't have the same limitations, and had different ideas about data storage and apis. That's why I wanted to make something new in parallel. It was a really hard decision to want to make something outside of 389 Directory Server (Because I really do love the project, and have great pride in the team), but I felt like it was going to be more productive to build in parallel, than ontop.</p> <h2 id="when-will-it-be-ready">When will it be ready?</h2> <p>I think that a single-server deployment will be usable for small installations early 2020, and a fully fledged system with replication would be late 2020. It depends on how much time I have and what parts I implement in what order. Current rough work order (late 2019) is indexing, RADIUS integration, claims, and then self-service/web ui.</p> OpenSUSE leap as a virtualisation host Mon, 02 Sep 2019 00:00:00 +0000 Unknown https://fy.blackhats.net.au/blog/2019-09-02-opensuse-leap-as-a-virtualisation-host/ https://fy.blackhats.net.au/blog/2019-09-02-opensuse-leap-as-a-virtualisation-host/ <h1 id="opensuse-leap-as-a-virtualisation-host">OpenSUSE leap as a virtualisation host</h1> <p>I've been rebuilding my network to use SUSE from CentOS, and the final server was my hypervisor. Most of the reason for this is the change in my employment, so I feel it's right to dogfood for my workplace.</p> <h2 id="what-you-will-need">What you will need</h2> <ul> <li>Some computer parts (assembaly may be required)</li> <li>OpenSUSE LEAP 15.1 media (dd if=opensuse.iso of=/dev/a_usb_i_hope)</li> </ul> <h2 id="what-are-we-aiming-for">What are we aiming for?</h2> <p>My new machine has dual NVME and dual 8TB spinning disks. The intent is to have the OS on the NVME and to have a large part of the NVME act as LVM cache for the spinning disk. The host won't run any applications beside libvirt, and has to connect a number of vlans over a link aggregation.</p> <h2 id="useful-commands">Useful commands</h2> <p>Through out this document I'll assume some details about your devices and partitions. To find your own, and to always check and confirm what you are doing, some command will help:</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>lsblk # Shows all block storage devices, and how (if) they are mounted. </span><span>lvs # shows all active logical volumes </span><span>vgs # shows all active volume groups </span><span>pvs # shows all active physical volumes </span><span>dmidecode # show hardware information </span><span>ls -al /dev/disk/by-&lt;ID TYPE&gt;/ # how to resolve disk path to a uuid etc. </span></code></pre> <p>I'm going to assume you have devices like:</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>/dev/nvme0 # the first nvme of your system that you install to. </span><span>/dev/nvme1 # the second nvme, used later </span><span>/dev/sda # Two larger block storage devices. </span><span>/dev/sdb </span></code></pre> <h2 id="install">Install</h2> <p>Install and follow the prompts. Importantly when you install you install to a single NVME, and choose transactional server + lvm, then put btrfs on the /root in the lvm. You want to partition such that there is free space still in the NVME - I left about 400GB unpartitioned for this. In other words, the disk should be like:</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>[ /efi | pv + vg system | pv (unused) ] </span><span> | /root (btrfs), /boot, /home | </span></code></pre> <p>Remember to leave about 1GB of freespace on the vg system to allow raid 1 metadata later!</p> <p>Honestly, it may take you a try or two to get this right with YaST, and it was probably the trickiest part of the install.</p> <p>You should also select that network management is via networkmanager, not wicked. You may want to enable ssh here. I disabled the firewall personally because there are no applications and it interfers with the bridging for the vms.</p> <h2 id="packages">Packages</h2> <p>Because this is suse transactional we need to add packages and reboot each time. Here is what I used, but you may find you don't need everything here:</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>transactional-update pkg install libvirt libvirt-daemon libvirt-daemon-qemu \ </span><span> sssd sssd-ad sssd-ldap sssd-tools docker zsh ipcalc python3-docker rdiff-backup \ </span><span> vim rsync iotop tmux fwupdate fwupdate-efi bridge-utils qemu-kvm apcupsd </span></code></pre> <p>Reboot, and you are ready to partition.</p> <h2 id="partitioning-post-install">Partitioning - post install</h2> <p>First, copy your gpt from the first NVME to the second. You can do this by hand with:</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>gdisk /dev/nvme0 </span><span>p </span><span>q </span><span> </span><span>gdisk /dev/nvme1 </span><span>c </span><span>&lt;duplicate the parameters as required&gt; </span></code></pre> <p>Now we'll make your /efi at least a little redundant</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>mkfs.fat /dev/nvme1p0 </span><span>ls -al /dev/disk/by-uuid/ </span><span># In the above, look for your new /efi fs, IE CE0A-2C1D -&gt; ../../nvme1n1p1 </span><span># Now add a line to /etc/fstab like: </span><span>UUID=CE0A-2C1D /boot/efi2 vfat defaults 0 0 </span></code></pre> <p>Now to really make this work, because it's transactional, you have to make a change to the /root, which is readonly! To do this run</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>transactional-update shell dup </span></code></pre> <p>This put's you in a shell at the end. Run:</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>mkdir /boot/efi2 </span></code></pre> <p>Now reboot. After the reboot your second efi should be mounted. rsync /boot/efi/* to /boot/efi2/. I leave it to the reader to decide how to sync this periodically.</p> <p>Next you can setup the raid 1 mirror for /root and the system vg.</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>pvcreate /dev/nvme1p1 </span><span>vgextend system /dev/nvme1p1 </span></code></pre> <p>Now we have enough pvs to make a raid 1, so we convert all our volumes:</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>lvconvert --type raid1 --mirrors 1 system/home </span><span>lvconvert --type raid1 --mirrors 1 system/root </span><span>lvconvert --type raid1 --mirrors 1 system/boot </span></code></pre> <p>If this fails with &quot;not enough space to alloc metadata&quot;, it's because you didn't leave space on the vg during install. Honestly, I made this mistake twice due to confusion about things leading to two reinstalls ...</p> <h2 id="getting-ready-to-cache">Getting ready to cache</h2> <p>Now lets get ready to cache some data. We'll make pvs and vgs for data:</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>pvcreate /dev/nvme0p2 </span><span>pvcreate /dev/nvme1p2 </span><span>pvcreate /dev/sda1 </span><span>pvcreate /dev/sda2 </span><span>vgcreate data /dev/nvme0p2 /dev/nvme1p2 /dev/sda1 /dev/sdb2 </span></code></pre> <p>Create the larger volume</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>lvcreate --type raid1 --mirrors 1 -L7.5T -n libvirt_t2 data /dev/sda1 /dev/sdb1 </span></code></pre> <p>Prepare the caches</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>lvcreate --type raid1 --mirrors 1 -L 4G -n libvirt_t2_meta data </span><span>lvcreate --type raid1 --mirrors 1 -L 400G -n libvirt_t2_cache data </span><span>lvconvert --type cache-pool --poolmetadata data/libvirt_t2_meta data/libvirt_t2_cache </span></code></pre> <p>Now put the caches in front of the disks. It's important for you to check you have the correct cachemode at this point, because you can't change it without removing and re-adding the cache. I choose writeback because my nvme devices are in a raid 1 mirror, and it helps to accelerate writes. You may err to use the default where the SSD's are read cache only.</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>lvconvert --type cache --cachemode writeback --cachepool data/libvirt_t2_cache data/libvirt_t2 </span><span>mkfs.xfs /dev/mapper/data-libvirt_t2 </span></code></pre> <p>You can monitor the amount of &quot;cached&quot; data in the data column of lvs.</p> <p>Now you can add this to /etc/fstab as any other xfs drive. I mounted it to /var/lib/libvirt/images.</p> <h2 id="network-manager">Network Manager</h2> <p>Now, I have to assemble the network bridges. Network Manager has some specific steps to follow to achieve this. I have:</p> <ul> <li>two intel gigabit ports</li> <li>the ports are link aggregated by 802.3ad</li> <li>there are multiple vlans ontop of the link agg</li> <li>bridges must be built on top of the vlans</li> </ul> <p>This requires a specific set of steps to layer this, because network manager sees the bridge and the lagg as seperate things that require the vlan to tie them together.</p> <p>Configure the link agg, and add our two ethernet phys</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>nmcli conn add type bond con-name bond0 ifname bond0 mode 802.3ad ipv4.method disabled ipv6.method ignore </span><span>nmcli connection add type ethernet con-name bond0-eth1 ifname eth1 master bond0 slave-type bond </span><span>nmcli connection add type ethernet con-name bond0-eth2 ifname eth2 master bond0 slave-type bond </span></code></pre> <p>Add a bridge for a vlan:</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>nmcli connection add type bridge con-name net_18 ifname net_18 ipv4.method disabled ipv6.method ignore </span></code></pre> <p>Now tie together a vlan on the bond, to the bridge we created.</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>nmcli connection add type vlan con-name bond0.18 ifname bond0.18 dev bond0 id 18 master net_18 slave-type bridge </span></code></pre> <p>You will need to repeat these last two commands as required for the vlans you have.</p> <h2 id="house-keeping">House Keeping</h2> <p>Finally you need to do some house keeping. Transactional server will automatically reboot and update so you need to be ready for this. You may disable this with:</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>systemctl disable transactional-update.timer </span></code></pre> <p>You likely want to edit:</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>/etc/sysconfig/libvirt-guests </span></code></pre> <p>To be able to handle guest shutdown policy due to a UPS failure or a reboot.</p> <p>Now you can enable and start libvirt:</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>systemctl enable libvirtd </span><span>systemctl start libvirtd </span></code></pre> <p>Finally you can build and import virtualmachines.</p> LDAP Filter Syntax Validation Thu, 29 Aug 2019 00:00:00 +0000 Unknown https://fy.blackhats.net.au/blog/2019-08-29-ldap-filter-syntax-validation/ https://fy.blackhats.net.au/blog/2019-08-29-ldap-filter-syntax-validation/ <h1 id="ldap-filter-syntax-validation">LDAP Filter Syntax Validation</h1> <p>Today I want to do a deep-dive into a change that will be released in 389 Directory Server 1.4.2. It's a reasonably complicated change for our server, but it has a simple user interaction for admins and developers. I want to peel back some of the layers to explain what kind of experience, thought and team work goes into a change like this.</p> <p>TL;DR - just keep upgrading your 389 Directory Server instance, and our 'correct by default' policy will apply, and you'll keep having the best LDAP server we can make :)</p> <h2 id="ldap-filters-and-how-they-work">LDAP Filters and How They Work</h2> <p><a href="/blog/html/pages/ldap_guide_part_3_filters.html">LDAP filters</a> are one of the primary methods of expression in LDAP, and are used in almost every aspect of the system - from finding who you are when you login, to asserting you are member of a group or have other security attributes.</p> <p>For the purposes of this discussion we'll look at this filter:</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>&#39;(|(cn=william)(cn=claire))&#39; </span></code></pre> <p>In order to execute these queries quickly (LDAP is designed to handle thousands of operations per second) we heavily rely on indexing. Indexing is often a topic where people believe it to be some kind of &quot;magic&quot; but it's reasonably simple: indexes are pre-computed partial result sets. So why do we need these?</p> <p>We'll imagine we have two entries (invalid, and truncated for brevity).</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>dn: cn=william,... </span><span>cn: william </span><span> </span><span>dn: cn=claire,... </span><span>cn: claire </span></code></pre> <p>These entries both have entry-ids - these id's are <em>per-server</em> in a replication group and are integers. You can show them by requesting entryid as an attribute in 389.</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>dn: cn=william,... </span><span>entryid: 1 </span><span>cn: william </span><span> </span><span>dn: cn=claire,... </span><span>entryid: 2 </span><span>cn: claire </span></code></pre> <p>Our entries are stored in the main-entry database in <em>/var/lib/dirsrv/slapd-standalone1/db/userRoot</em> in the file &quot;id2entry.db4&quot;. This is a key-value database where the keys are the entryid, and the value is the serialised entry itself. Roughly, it's:</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>[ ID ][ Entry ] </span><span> 1 dn: cn=william,... </span><span> cn: william </span><span> </span><span> 2 dn: cn=claire,... </span><span> cn: claire </span></code></pre> <p>Now, if we had NO indexes, to evaluate our filters we have to scan every entry of id2entry to determine if the filter matches. This algorithm is:</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>candidate_set = [] </span><span>for id in id-min to id-max: </span><span> entry = load_entry_by_id(id) </span><span> if apply_filter(filter, entry): </span><span> candidate_set.append(entry) </span></code></pre> <p>For two entries, this may be fast, but when you have 1000, 10.000, or even millions, this is extremely slow. We call these searches <em>full table scans</em> or in 389 DS, <em>ALLIDS</em> searches.</p> <p>To make our searches faster we have indexes. An index is a mapping of a partial query term to an id list (In 389 we call these IDLs). An IDL is a set of integers. Our index for these examples would likely be something like:</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>cn </span><span>=william: [1, ] </span><span>=claire: [2, ] </span></code></pre> <p>These indexes are also stored in key-value databases in userRoot - you can see this as cn.db4.</p> <p>So when we have an indexed term, to evaluate the query, we'll load the indexes, then using mathematical set operations, we then produce a candidate_id_set, and we can then load the entries that only match.</p> <p>For example in psuedo python code:</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span># Assume query is: (cn=william) </span><span> </span><span>attr = filter.get_attr_name() </span><span>with open(&#39;%s.db&#39; % attr) as index: </span><span> idl = index.get(&#39;=william&#39;) # from the filter :) </span><span> </span><span>for id in idl: </span><span> ... # as before. </span></code></pre> <p>So we can see now that when we load the idl for cn index, this would give us the set [1, ]. Even if the database had 100 million entries, as our idl is a single value, we only need to load the one entry that matches. Neat!</p> <p>When we have a more complex operation such as AND and OR, we can now manipulate the idl sets. For example:</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>(|(uid=claire)(uid=william)) </span><span> uid =claire -&gt; idl [2, ] </span><span> uid =william -&gt; idl [1, ] </span><span> </span><span>candidate_idl_set = union([2, ], [1, ]) </span><span># [1, 2] </span></code></pre> <p>This means again, even with millions of entries, we only need to load entry 1 and 2 to uphold the query provided to us.</p> <p>So we finally know enough to understand how our example query is executed. PHEW!</p> <h2 id="unindexed-attributes">Unindexed Attributes</h2> <p>However, it's not always so easy. When we have an attribute that isn't indexed, we have to handle this situation. In these cases, while we operate on the idl set, we may insert an idl with the value of ALLIDS (which as previously mentioned, is the &quot;set of all entries&quot;). This can have various effects.</p> <p>If this was an AND query, we can annotate that the filter is <em>partially</em> resolved. This means that if we had:</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>(&amp;(cn=william)(unindexed=foo)) </span></code></pre> <p>Because an AND condition, both filter components must be satisfied, we have a partial candidate set from cn=william of [1, ]. We can load this partial candidate set, and then apply the filter test as in the full table scan case, but as we only apply it to a single entry this is really fast.</p> <p>The real problem is OR queries. If we had:</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>(|(cn=william)(unindexed=foo)) </span></code></pre> <p>Because OR means that both filter components <em>could</em> be satisfied, we have to turn unindexd into ALLIDS, and the result of the OR as a whole is ALLIDS. So even if we have 30 indexed values in the OR, a single ALLIDS (unindexed value) will always turn that OR into a full table scan. This is not good for performance!</p> <h2 id="missing-attributes">Missing Attributes</h2> <p>So as a weirder case ... what if the attribute doesn't exist in schema at all? For example we could search for Microsoft AD attributes in 389 Directory Server, or we could submit bogus filters like &quot;(whargarble=foo)&quot;. What happens here?</p> <p>Well, historically we treated these the same as <em>unindexed</em> queries. Which means that any term that is not in schema, would be treated as ALLIDS. This led to a &quot;quitely known&quot; denial of service attack again 389 Directory Server where you could emit a large number of queries for attributes that don't exist, causing the server to attempt many ALLIDS scans. We have some defences like the allids limit (how many entries you can full table scan before giving up). But it can still cause entry cache churn and other performance issues.</p> <p>I was first made aware of this issue in 2014 while working for University of Adelaide where our VMWare service would query LDAP for MS attributes, causing a large performance issue. We resolved this by adding the MS attributes to schema and indexing them so that they would create empty indexes - now we would call this in 389 Directory Server and &quot;idl_alloc(0)&quot; or &quot;empty IDL&quot;.</p> <p>When initially hired by Red Hat in 2015 I knew this issue existed but I didn't know enough about the server core to really fix it, so it went in the back of my mind ... it was rare to have a customer raise this issue, but we had the work around and so I was able to advise support services on how to mitigate this.</p> <p>In 2019 however, while investigating an issue related to filter optimisation, I was made aware of an issue with FreeIPA where they were doing certmap queries that requested MS Cert attributes. However it would cause large performance issues. We now had the issue again, and in a large widely installed product so it was time to tackle it.</p> <h2 id="how-to-handle-this">How to handle this?</h2> <p>A major issue in this is &quot;never breaking customers&quot;. Because we had always supported this behaviour there is a risk that any solution would cause customer queries to &quot;silently&quot; begin to break if we issued a fix or change. More accurately, any change to how we execute the filters could cause results of the filters to change, which would disrupt customers.</p> <p>Saying this, there is also precedent that 389 Directory Server was handling this incorrectly. From the RFC for LDAP it was noted:</p> <p><em>Any assertion about the values of such an attribute is only defined if the AttributeType is known by the evaluating mechanism, the purported AttributeValue(s) conforms to the attribute syntax defined for that attribute type, the implied or indicated matching rule is applicable to that attribute type, and (when used) a presented matchValue conforms to the syntax defined for the indicated matching rules. When these conditions are not met, the FilterItem shall evaluate to the logical value UNDEFINED. An assertion which is defined by these conditions additionally evaluates to UNDEFINED if it relates to an attribute value and the attribute type is not present in an attribute against which the assertion is being tested. An assertion which is defined by these conditions and relates to the presence of an attribute type evaluates to FALSE.</em></p> <p>Translation: If a filter component (IE nonexist=foo) is in a filter but NOT in the schema, the result of the filter's evaluation is an empty-set aka undefined.</p> <p>It was also clear that if an engaged and active consumer like FreeIPA is making this mistake, then it must be overlooked by many others without notice. So there is sometimes value in helping to raise the standard so that everyone benefits, and highlight mistakes quicker.</p> <h2 id="the-technical-solution">The Technical Solution</h2> <p>This is the easy part - we add a new configuration option with three states. &quot;on&quot;, &quot;off&quot;, &quot;warn&quot;. &quot;On&quot; would enable the strictest handling of filters, rejecting them an not continuing if any attribute requested was not in the schema. &quot;Warn&quot; would provide the rfc compliant behaviour, mapping to empty-set index, and notifying in the logs that this occured. Finally, &quot;off&quot; would be the previous &quot;silently allow&quot; behaviour.</p> <p>This was easily achieved in filter parsing, by checking the attribute of each filter component against our schema hashmap. We then tag the filter element, and depending on the current setting level reject or continue.</p> <p>In the query execution code, we now check the filter tag to understand if the attribute is schema present or not. If it's flagged as &quot;undefined&quot;, then we immediately shortcut to return idl_alloc(0) instead of returning ALLIDS on the failure to find the relevant index db.</p> <p>We can show the performance impact of this change:</p> <p>Before with non-existant attribute</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>Average rate: 7.40/thr </span></code></pre> <p>After with &quot;warn&quot; enabled (map to empty set)</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>Average rate: 4808.70/thr </span></code></pre> <p>This is a huge improvement, and certainly shows the risk of DOS and how effective the solution was!</p> <h2 id="the-social-solution">The Social Solution</h2> <p>Of course, this is the hard part - the 389 Directory Server team are all amazingly smart people, from many countries, and all live around the world. They all care that the server is the best possible, and that our standards as a team are high. That means when introducing a change that has a risk of affecting query result sets like this, they pay attention, and ask many difficult questions about how the feature will be implemented.</p> <p>The first important justification - is a change like this worth while? We can see from the performance results that the risk of DOS is reasonable, so the answer there becomes Yes from a security view. But it's also important to consider the cost on consumers - is this change going to benefit FreeIPA for example? As I am biased being the author I want to say &quot;yes&quot; - by notifying or rejecting invalid filters earlier, we can help FreeIPA developers improve their code quality, without expecting them to know LDAP inside and out.</p> <p>The next major question is performance - before the feature was developed there is clearly a risk of DOS, but when we implement this we are performing additional locking on the schema. Is that a risk to our standalone performance or normal operating conditions. This had to also be discussed and assessed.</p> <p>A really important point that was raised by Thierry is how we communicated these errors too. Previously we would use the &quot;notes=&quot; field of the access log. It looks like this:</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>conn=1 op=4 RESULT err=0 tag=101 nentries=13 etime=0.0003795424 notes=U </span></code></pre> <p>The challenge with the notes= field, is that it's easy to overlook, and unless you are familar, hard to see what this is indicating. In this case, notes=U means partially unindexed query (one filter component but not all returned ALLIDS).</p> <p>We can't change the notes field due to the risk of breaking our own scripts like logconv.pl, support tools developed by RH or SUSE, or even integrations to platforms like splunk. But clearly we need a way to detail what is happening with your filter. So Thierry suggested an extension to have details about the provided notes. Now we get:</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>conn=1 op=4 RESULT err=0 tag=101 nentries=13 etime=0.0003795424 notes=U details=&quot;Partially Unindexed Filter&quot; </span><span>conn=1 op=8 RESULT err=0 tag=101 nentries=0 etime=0.0001886208 notes=F details=&quot;Filter Element Missing From Schema&quot; </span></code></pre> <p>So we have extended our log message, but without breaking existing integrations.</p> <p>The final question is what our defaults should be. It's one thing to have this feature, but what should we ship with? Do we strictly reject filters? Warn? Or disable, and expect people to turn this on.</p> <p>This became a long discussion with Ludwig, Thierry and I - we discussed the risk of DOS in the first place, what the impact of the levels could be, how it could break legacy applications or sites using deprecated features or with weird data imports. Many different aspects were considered. We decided to default to &quot;warn&quot; (non-existant becomes empty-set), and we settled on communication with support to advise them of the upcoming change, but also we considered that our &quot;back out&quot; plan is to change the default and ship a patch if there is a large volume of negative feedback.</p> <h2 id="conclusion">Conclusion</h2> <p>As of today, the PR is merged, and the code on it's way to the next release. It's a long process but the process exists to ensure we do what's best for our users, while we try to balance many different aspects. We have a great team of people, with decades of experience from many backgrounds which means that these discussions can be long and detailed, but in the end, we hope to give what is the best product possible to our community.</p> <p>It's also valuable to share how much thought and effort goes into projects - in your life you may only interact with 1% of our work through our configuration and system, but we have an iceberg of decisions and design process that affects you every day, where we have to be responsible and considerate in our actions.</p> <p>I hope you enjoyed this exploration of this change!</p> <h2 id="references">References</h2> <p><a href="https://pagure.io/389-ds-base/pull-request/50379">PR#50379</a></p> Using ramdisks with Cargo Mon, 26 Aug 2019 00:00:00 +0000 Unknown https://fy.blackhats.net.au/blog/2019-08-26-using-ramdisks-with-cargo/ https://fy.blackhats.net.au/blog/2019-08-26-using-ramdisks-with-cargo/ <h1 id="using-ramdisks-with-cargo">Using ramdisks with Cargo</h1> <p>I have a bit of a history of killing SSDs - probably because I do a bit too much compiling and management of thousands of tiny files. Plenty of developers have this problem! So while thinking one evening, I was curious if I could setup a ramdisk on my mac for my cargo work to output to.</p> <h2 id="making-the-ramdisk">Making the ramdisk</h2> <p>On Linux you'll need to use tmpfs or some access to /dev/shm.</p> <p>On OSX you need to run a script like the following:</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>diskutil partitionDisk $(hdiutil attach -nomount ram://4096000) 1 GPTFormat APFS &#39;ramdisk&#39; &#39;100%&#39; </span></code></pre> <p>This creates and mounts a 4GB ramdisk to /Volumes/ramdisk. Make sure you have enough ram!</p> <h2 id="asking-cargo-to-use-it">Asking cargo to use it</h2> <p>We probably don't want to make our changes permant in Cargo.toml, so we'll use the environment:</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>CARGO_TARGET_DIR=/Volumes/ramdisk/rs cargo ... </span></code></pre> <h2 id="does-it-work">Does it work?</h2> <p>Yes!</p> <p>Disk Build (SSD, 2018MBP)</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>Finished dev [unoptimized + debuginfo] target(s) in 2m 29s </span></code></pre> <p>4 GB APFS ramdisk</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>Finished dev [unoptimized + debuginfo] target(s) in 1m 53s </span></code></pre> <p>For me it's more valuable to try and save those precious SSD write cycles, so I think I'll try to stick with this setup. You can see how much rust writes by doing a clean + build. My project used the following:</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>Filesystem Size Used Avail Capacity iused ifree %iused Mounted on </span><span>/dev/disk110s1 2.0Gi 1.2Gi 751Mi 63% 3910 9223372036854771897 0% /Volumes/ramdisk </span></code></pre> <h2 id="make-it-permanent">Make it permanent</h2> <p>Put the following in /Library/LaunchDaemons/au.net.blackhats.fy.ramdisk.plist</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>&lt;?xml version=&quot;1.0&quot; encoding=&quot;UTF-8&quot;?&gt; </span><span>&lt;!DOCTYPE plist PUBLIC &quot;-//Apple//DTD PLIST 1.0//EN&quot; &quot;http://www.apple.com/DTDs/PropertyList-1.0.dtd&quot;&gt; </span><span>&lt;plist version=&quot;1.0&quot;&gt; </span><span> &lt;dict&gt; </span><span> &lt;key&gt;Label&lt;/key&gt; </span><span> &lt;string&gt;au.net.blackhats.fy.ramdisk&lt;/string&gt; </span><span> &lt;key&gt;Program&lt;/key&gt; </span><span> &lt;string&gt;/usr/local/libexec/ramdisk.sh&lt;/string&gt; </span><span> &lt;key&gt;RunAtLoad&lt;/key&gt; </span><span> &lt;true/&gt; </span><span> &lt;key&gt;StandardOutPath&lt;/key&gt; </span><span> &lt;string&gt;/var/log/ramdisk.log&lt;/string&gt; </span><span> &lt;/dict&gt; </span><span>&lt;/plist&gt; </span></code></pre> <p>And the following into /usr/local/libexec/ramdisk.sh</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>#!/bin/bash </span><span>date </span><span>diskutil partitionDisk $(hdiutil attach -nomount ram://4096000) 1 GPTFormat APFS &#39;ramdisk&#39; &#39;100%&#39; </span></code></pre> <p>Finally put this in your <a href="https://doc.rust-lang.org/cargo/reference/config.html">cargo file of choice</a></p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>[build] </span><span>target-dir = &quot;/Volumes/ramdisk/rs&quot; </span></code></pre> <p>Future william will need to work out if there are negative consequences to multiple cargo projects sharing the same target directory ... hope not!</p> <h2 id="launchctl-tips">Launchctl tips</h2> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span># Init the service </span><span>launchctl load /Library/LaunchDaemons/au.net.blackhats.fy.ramdisk.plist </span></code></pre> <h2 id="references">References</h2> <p><a href="https://www.launchd.info/">lanuchd.info</a></p> CPU atomics and orderings explained Tue, 16 Jul 2019 00:00:00 +0000 Unknown https://fy.blackhats.net.au/blog/2019-07-16-cpu-atomics-and-orderings-explained/ https://fy.blackhats.net.au/blog/2019-07-16-cpu-atomics-and-orderings-explained/ <h1 id="cpu-atomics-and-orderings-explained">CPU atomics and orderings explained</h1> <p>Sometimes the question comes up about how CPU memory orderings work, and what they do. I hope this post explains it in a really accessible way.</p> <h2 id="short-version-i-wanna-code">Short Version - I wanna code!</h2> <p>Summary - The memory model you commonly see is from C++ and it defines:</p> <ul> <li>Relaxed</li> <li>Acquire</li> <li>Release</li> <li>Acquire/Release (sometimes AcqRel)</li> <li>SeqCst</li> </ul> <p>There are memory orderings - every operation is &quot;atomic&quot;, so will work correctly, but there rules define how the memory and code <em>around</em> the atomic are influenced.</p> <p>If in doubt - use SeqCst - it's the strongest guarantee and prevents all re-ordering of operations and will do the right thing.</p> <p>The summary is:</p> <ul> <li>Relaxed - no ordering guarantees, just execute the atomic as is.</li> <li>Acquire - all code after this atomic, will be executed after the atomic.</li> <li>Release - all code before this atomic, will be executed before the atomic.</li> <li>Acquire/Release - both Acquire and Release - ie code stays before and after.</li> <li>SeqCst - Stronger consistency of Acquire/Release.</li> </ul> <h2 id="long-version-let-s-begin">Long Version ... let's begin ...</h2> <p>So why do we have memory and operation orderings at all? Let's look at some code to explain:</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>let mut x = 0; </span><span>let mut y = 0; </span><span>x = x + 3; </span><span>y = y + 7; </span><span>x = x + 4; </span><span>x = y + x; </span></code></pre> <p>Really trivial example - now to us as a human, we read this and see a set of operations that are linear by time. That means, they execute from top to bottom, in order.</p> <p>However, this is not how computers work. First, compilers will optimise your code, and optimisation means re-ordering of the operations to achieve better results. A compiler may optimise this to:</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>let mut x = 0; </span><span>let mut y = 0; </span><span>// Note removal of the x + 3 and x + 4, folded to a single operation. </span><span>x = x + 7 </span><span>y = y + 7; </span><span>x = y + x; </span></code></pre> <p>Now there is a second element. Your CPU presents the illusion of running as a linear system, but it's actually an asynchronous, out-of-order task execution engine. That means a CPU will reorder your instructions, and may even run them concurrently and asynchronously.</p> <p>For example, your CPU will have both x + 7 and y + 7 in the pipeline, even though neither operation has completed - they are effectively running at the &quot;same time&quot; (concurrently).</p> <p>When you write a single thread program, you generally won't notice this behaviour. This is because a lot of smart people write compilers and CPU's to give the illusion of linear ordering, even though both of them are operating very differently.</p> <p>Now we want to write a multithreaded application. Suddenly this is the challenge:</p> <p><em>We write a concurrent program, in a linear language, executed on a concurrent asynchronous machine.</em></p> <p>This means there is a challenge is the translation between our mind (thinking about the concurrent problem), the program (which we have to express as a linear set of operations), which then runs on our CPU (an async concurrent device).</p> <p>Phew. How do computers even work in this scenario?!</p> <h2 id="why-are-cpu-s-async">Why are CPU's async?</h2> <p>CPU's have to be async to be fast - remember spectre and meltdown? These are attacks based on measuring the side effects of CPU's asynchronous behaviour. While computers are &quot;fast&quot; these attacks will always be possible, because to make a CPU synchronous is <em>slow</em> - and asynchronous behaviour will always have measurable side effects. Every modern CPU's performance is an illusion of async forbidden magic.</p> <p>A large portion of the async behaviour comes from the interaction of the CPU, cache, and memory.</p> <p>In order to provide the &quot;illusion&quot; of a coherent synchronous memory interface there is no seperation of your programs cache and memory. When the cpu wants to access &quot;memory&quot; the CPU cache is utilised transparently and will handle the request, and only on a cache miss, will we retrieve the values from RAM.</p> <p>(Aside: in almost all cases more CPU cache, not frequency will make your system perform better, because a cache miss will mean your task stalls waiting on RAM. Ohh no!)</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>CPU -&gt; Cache -&gt; RAM </span></code></pre> <p>When you have multiple CPU's, each CPU has it's own L1 cache:</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>CPU1 -&gt; L1 Cache -&gt; | | </span><span>CPU2 -&gt; L1 Cache -&gt; | Shared L2/L3 | -&gt; RAM </span><span>CPU3 -&gt; L1 Cache -&gt; | | </span><span>CPU4 -&gt; L1 Cache -&gt; | | </span></code></pre> <p>Ahhh! Suddenly we can see where problems can occur - each CPU has an L1 cache, which is transparent to memory but <em>unique</em> to the CPU. This means that each CPU can make a change to the same piece of memory in their L1 cache <em>without the other CPU knowing</em>. To help explain, let's show a demo.</p> <h2 id="cpu-just-trash-my-variables">CPU just trash my variables</h2> <p>We'll assume we now have two threads - my code is in rust again, and there is a good reason for the unsafes - this code really is unsafe!</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>// assume global x: usize = 0; y: usize = 0; </span><span> </span><span>THREAD 1 THREAD 2 </span><span> </span><span>if unsafe { *x == 1 } { unsafe { </span><span> unsafe { *y += 1 } *y = 10; </span><span>} *x = 1; </span><span> } </span></code></pre> <p>At the end of execution, what state will X and Y be in? The answer is &quot;it depends&quot;:</p> <ul> <li>What order did the threads run?</li> <li>The state of the L1 cache of each CPU</li> <li>The possible interleavings of the operations.</li> <li>Compiler re-ordering</li> </ul> <p>In the end the result of x will always be 1 - because x is only mutated in one thread, the caches will &quot;eventually&quot; (explained soon) become consistent.</p> <p>The real question is y. y could be:</p> <ul> <li>10</li> <li>11</li> <li>1</li> </ul> <p><em>10</em> - This can occur because in thread 2, x = 1 is re-ordered above y = 10, causing the thread 1 &quot;y += 1&quot; to execute, followed by thread 2 assign 10 directly to y. It can also occur because the check for x == 1 occurs first, so y += 1 is skipped, then thread 2 is run, causing y = 10. Two ways to achieve the same result!</p> <p><em>11</em> - This occurs in the &quot;normal&quot; execution path - all things considered it's a miracle :)</p> <p><em>1</em> - This is the most complex one - The y = 10 in thread 2 is applied, but the result is never sent to THREAD 1's cache, so x = 1 occurs and <em>is</em> made available to THREAD 1 (yes, this is possible to have different values made available to each cpu ...). Then thread 1 executes y (0) += 1, which is then sent back trampling the value of y = 10 from thread 2.</p> <p>If you want to know more about this and many other horrors of CPU execution, Paul McKenny is an expert in this field and has many talks at LCA and others on the topic. He can be found on <a href="https://twitter.com/paulmckrcu">twitter</a> and is super helpful if you have questions.</p> <h2 id="so-how-does-a-cpu-work-at-all">So how does a CPU work at all?</h2> <p>Obviously your system (likely a multicore system) works today - so it must be possible to write correct concurrent software. Cache's are kept in sync via a protocol called MESI. This is a state machine describing the states of memory and cache, and how they can be synchronised. The states are:</p> <ul> <li>Modified</li> <li>Exclusive</li> <li>Shared</li> <li>Invalid</li> </ul> <p>What's interesting about MESI is that each cache line is maintaining it's own state machine of the memory addresses - it's not a global state machine. To coordinate CPU's asynchronously message each other.</p> <p>A CPU can be messaged via IPC (Inter-Processor-Communication) to say that another CPU wants to &quot;claim&quot; exclusive ownership of a memory address, or to indicate that it has changed the content of a memory address and you should discard your version. It's important to understand these messages are <em>asynchronous</em>. When a CPU modifies an address it does not immediately send the invalidation message to all other CPU's - and when a CPU recieves the invalidation request it does not immediately act upon that message.</p> <p>If CPU's did &quot;synchronously&quot; act on all these messages, they would be spending so much time handling IPC traffic, they would never get anything done!</p> <p>As a result, it must be possible to indicate to a CPU that it's time to send or acknowledge these invalidations in the cache line. This is where barriers, or the memory orderings come in.</p> <ul> <li>Relaxed - No messages are sent or acknowledged.</li> <li>Release - flush all pending invalidations to be sent to other CPUS</li> <li>Acquire - Acknowledge and process all invalidation requests in my queue</li> <li>Acquire/Release - flush all outgoing invalidations, and process my incomming queue</li> <li>SeqCst - as AcqRel, but with some other guarantees around ordering that are beyond this discussion.</li> </ul> <h2 id="understand-a-mutex">Understand a Mutex</h2> <p>With this knowledge in place, we are finally in a position to understand the operations of a Mutex</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>// Assume mutex: Mutex&lt;usize&gt; = Mutex::new(0); </span><span> </span><span>THREAD 1 THREAD 2 </span><span> </span><span>{ { </span><span> let guard = mutex.lock() let guard = mutex.lock() </span><span> *guard += 1; println!(*guard) </span><span>} } </span></code></pre> <p>We know very clearly that this will print 1 or 0 - it's safe, no weird behaviours. Let's explain this case though:</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>THREAD 1 </span><span> </span><span>{ </span><span> let guard = mutex.lock() </span><span> // Acquire here! </span><span> // All invalidation handled, guard is 0. </span><span> // Compiler is told &quot;all following code must stay after .lock()&quot;. </span><span> *guard += 1; </span><span> // content of usize is changed, invalid req is queue </span><span>} </span><span>// Release here! </span><span>// Guard goes out of scope, invalidation reqs sent to all CPU&#39;s </span><span>// Compiler told all proceeding code must stay above this point. </span><span> </span><span> THREAD 2 </span><span> </span><span> { </span><span> let guard = mutex.lock() </span><span> // Acquire here! </span><span> // All invalidations handled - previous cache of usize discarded </span><span> // and read from THREAD 1 cache into S state. </span><span> // Compiler is told &quot;all following code must stay after .lock()&quot;. </span><span> println(*guard); </span><span> } </span><span> // Release here! </span><span> // Guard goes out of scope, no invalidations sent due to </span><span> // no modifications. </span><span> // Compiler told all proceeding code must stay above this point. </span></code></pre> <p>And there we have it! How barriers allow us to define an ordering in code and a CPU, to ensure our caches and compiler outputs are correct and consistent.</p> <h2 id="benefits-of-rust">Benefits of Rust</h2> <p>A nice benefit of Rust, and knowing these MESI states now, we can see that the best way to run a system is to minimise the number of invalidations being sent and acknowledged as this always causes a delay on CPU time. Rust variables are always mutable or immutable. These map almost directly to the E and S states of MESI. A mutable value is always exclusive to a single cache line, with no contention - and immutable values can be placed into the Shared state allowing each CPU to maintain a cache copy for higher performance.</p> <p>This is one of the reasons for Rust's amazing concurrency story is that the memory in your program map to cache states very clearly.</p> <p>It's also why it's unsafe to mutate a pointer between two threads (a global) - because the cache of the two cpus' won't be coherent, and you may not cause a crash, but one threads work will absolutely be lost!</p> <p>Finally, it's important to see that this is why using the correct concurrency primitives matter -it can highly influence your cache behaviour in your program and how that affects cache line contention and performance.</p> <p>For comments and more, please feel free to <a href="mailto:william@blackhats.net.au">email me!</a></p> <h2 id="shameless-plug">Shameless Plug</h2> <p>I'm the author and maintainer of Conc Read - a concurrently readable datastructure library for Rust. <a href="https://crates.io/crates/concread">Check it out on crates.io!</a></p> <h2 id="references">References</h2> <p><a href="https://people.freebsd.org/~lstewart/articles/cpumemory.pdf">What every programmer should know about memory (pdf)</a></p> <p><a href="https://doc.rust-lang.org/nomicon/atomics.html">Rust-nomicon - memory ordering</a></p> <p><a href="https://gamozolabs.github.io/metrology/2019/08/19/sushi_roll.html">Microarchitectural inspection with Sushi Roll</a></p> I no longer recommend FreeIPA Wed, 10 Jul 2019 00:00:00 +0000 Unknown https://fy.blackhats.net.au/blog/2019-07-10-i-no-longer-recommend-freeipa/ https://fy.blackhats.net.au/blog/2019-07-10-i-no-longer-recommend-freeipa/ <h1 id="i-no-longer-recommend-freeipa">I no longer recommend FreeIPA</h1> <p>It's probably taken me a few years to write this, but I can no longer recommend FreeIPA for IDM installations.</p> <h2 id="why-not">Why not?</h2> <p>The FreeIPA project focused on Kerberos and SSSD, with enough other parts glued on to look like a complete IDM project. Now that's fine, but it means that concerns in other parts of the project are largely ignored. It creates design decisions that are not scalable or robust.</p> <p>Due to these decisions IPA has stability issues and scaling issues that other products do not.</p> <p>To be clear: security systems like IDM or LDAP can <em>never</em> go down. That's not acceptable.</p> <h2 id="what-do-you-recommend-instead">What do you recommend instead?</h2> <ul> <li><a href="https://kanidm.github.io/kanidm/stable/">Kanidm</a></li> <li>Samba with AD</li> <li>AzureAD</li> <li>389 Directory Server</li> </ul> <p>All of these projects are very reliable, secure, scalable.</p> <p>Kanidm specifically is my own project built from the ground up, which has tried to learn from the mistakes and successes of AD, FreeIPA and 389-ds. It's what I recommend the most.</p> <p>Additionally, we have done a lot of work into 389 to improve our out of box IDM capabilities too, but there is more to be done too. The Samba AD team have done great things too, and deserve a lot of respect for what they have done.</p> <h2 id="is-there-more-detail-than-this">Is there more detail than this?</h2> <p>Yes - buy me a drink and I'll talk :)</p> <h2 id="didn-t-you-help">Didn't you help?</h2> <p>I tried and it was not taken on board. Issues I reported years ago still exist today 🙃.</p> Using 389ds with docker Fri, 05 Jul 2019 00:00:00 +0000 Unknown https://fy.blackhats.net.au/blog/2019-07-05-using-389ds-with-docker/ https://fy.blackhats.net.au/blog/2019-07-05-using-389ds-with-docker/ <h1 id="using-389ds-with-docker">Using 389ds with docker</h1> <p>I've been wanting to containerise 389 Directory Server for a long time - it's been a long road to get here, but I think that our container support is getting very close to a production ready and capable level. It took so long due to health issues and generally my obsession to do everything right.</p> <p>Today, container support along with our new command line tools makes 389 a complete breeze to administer. So lets go through an example of a deployment now.</p> <p>Please note: the container image here is a git-master build and is not production ready as of 2019-07, hopefully this changes soon.</p> <h2 id="getting-the-container">Getting the Container</h2> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>docker pull firstyear/389ds:latest </span></code></pre> <p>If you want to run an ephemeral instance (IE you will LOSE all your data on a restart)</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>docker run firstyear/389ds:latest </span></code></pre> <p>If you want your data to persist, you need to attach a volume at /data:</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>docker volume create 389ds_data </span><span>docker run -v 389ds_data:/data firstyear/389ds:latest </span></code></pre> <p>The image exposes ports 3389 and 3636, so you may want to consider publishing these if you want external access.</p> <p>The container should now setup and run an instance! That's it, LDAP has never been easier to deploy!</p> <h2 id="actually-adding-some-data">Actually Adding Some Data ...</h2> <p>LDAP only really matters if we have some data! So we'll create a new backend. You need to run these instructions inside the current container, so I prefix these with:</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>docker exec -i -t &lt;name of container&gt; &lt;command&gt; </span><span>docker exec -i -t 389inst dsconf .... </span></code></pre> <p>This uses the ldapi socket via /data, and authenticates you based on your process uid to map you to the LDAP administrator account - basically, it's secure, host only admin access to your data.</p> <p>Now you can choose any suffix you like, generally based on your dns name (IE I use dc=blackhats,dc=net,dc=au).</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>dsconf localhost backend create --suffix dc=example,dc=com --be-name userRoot </span><span>&gt; The database was sucessfully created </span></code></pre> <p>Now fill in the suffix details into your configuration of the container. You'll need to find where docker stores the volume on your host for this (docker inspect will help you). My location is listed here:</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>vim /var/lib/docker/volumes/389ds_data/_data/config/container.inf </span><span> </span><span>--&gt; change </span><span># basedn ... </span><span>--&gt; to </span><span>basedn = dc=example,dc=com </span></code></pre> <p>Now you can populate data into that: The dsidm command is our tool to manage users and groups of a backend, and it can provide initialised data which has best-practice aci's, demo users and groups and starts as a great place for you to build an IDM system.</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>dsidm localhost initialise </span></code></pre> <p>That's it! You can now see you have a user and a group!</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>dsidm localhost user list </span><span>&gt; demo_user </span><span>dsidm localhost group list </span><span>&gt; demo_group </span></code></pre> <p>You can create your own user:</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>dsidm localhost user create --uid william --cn William --displayName &#39;William Brown&#39; --uidNumber 1000 --gidNumber 1000 --homeDirectory /home/william </span><span>&gt; Successfully created william </span><span>dsidm localhost user get william </span></code></pre> <p>It's trivial to add an ssh key to the user:</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>dsidm localhost user modify william add:nsSshPublicKey:AAA... </span><span>&gt; Successfully modified uid=william,ou=people,dc=example,dc=com </span></code></pre> <p>Or to add them to a group:</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>dsidm localhost group add_member demo_group uid=william,ou=people,dc=example,dc=com </span><span>&gt; added member: uid=william,ou=people,dc=example,dc=com </span><span>dsidm localhost group members demo_group </span><span>&gt; dn: uid=william,ou=people,dc=example,dc=com </span></code></pre> <p>Finally, we can even generale config templates for your applications:</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>dsidm localhost client_config sssd.conf </span><span>dsidm localhost client_config ldap.conf </span><span>dsidm localhost client_config display </span></code></pre> <p>I'm happy to say, LDAP administration has never been easier - we plan to add more functionality to enabled broader ranges of administrative tasks, especially in the IDM area and management of the configuration. It's honestly hard to beleve that in a shortlist of commands you can now have a fully functional LDAP IDM solution working.</p> Implementing Webauthn - a series of complexities ... Sun, 28 Apr 2019 00:00:00 +0000 Unknown https://fy.blackhats.net.au/blog/2019-04-28-implementing-webauthn-a-series-of-complexities/ https://fy.blackhats.net.au/blog/2019-04-28-implementing-webauthn-a-series-of-complexities/ <h1 id="implementing-webauthn-a-series-of-complexities">Implementing Webauthn - a series of complexities ...</h1> <p>I have recently started to work on a rust webauthn library, to allow servers to be implemented. However, in this process I have noticed a few complexities to an API that should have so much promise for improving the state of authentication. So far I can say I have not found any cryptographic issues, but the design of the standard does raise questions about the ability for people to correctly implement Webauthn servers.</p> <h2 id="odd-structure-decisions">Odd structure decisions</h2> <p>Webauth is made up of multiple encoding standards. There is a good reason for this, which is that the json parts are for the webbrowser, and the cbor parts are for ctap and the authenticator device.</p> <p>However, I quickly noticed an issue in the Attestation Object, as described here <a href="https://w3c.github.io/webauthn/#sctn-attestation">https://w3c.github.io/webauthn/#sctn-attestation</a> . Can you see the problem?</p> <p>The problem is that the Authenticator Data relies on hand-parsing bytes, and has two structures that are concatenated with no length. This means:</p> <ul> <li>You have to hand parse bytes 0 -&gt; 36</li> <li>You then have to CBOR deserialise the Attested Cred Data (if present)</li> <li>You then need to serialise the ACD back to bytes and record that length (if your library doesn't tell you how long the amount of data is parsed was).</li> <li>Then you need to CBOR deserialise the Extensions.</li> </ul> <p>What's more insulting about this situation is that the Authenticator Data literally is part of the AttestationObject which is already provided as CBOR! There seems to be no obvious reason for this to require hand-parsing, as the Authenticator Data which will be signature checked, has it's byte form checked, so you could have the AttestationObject store authDataBytes, then you can CBOR decode the nested structure (allowing the hashing of the bytes later).</p> <p>There are many risks here because now you have requirements to length check all the parameters which people could get wrong - when CBOR would handle this correctly for you you and provides a good level of correctness that the structure is altered. I also trust the CBOR parser authors to do proper length checks too compared to my crappy byte parsing code!</p> <h2 id="confusing-naming-conventions-and-layout">Confusing Naming Conventions and Layout</h2> <p>The entire standard is full of various names and structures, which are complex, arbitrarily nested and hard to see why they are designed this way. Perhaps it's a legacy compatability issue? More likely I think it's object-oriented programming leaking into the specification, which is a paradigm that is not universally applicable.</p> <p>Regardless, it would be good if the structures were flatter, and named better. There are many confusing structure names throughout the standard, and it can sometimes be hard to identify what you require and don't require.</p> <p>Additionally, naming of fields and their use, uses abbrivations to save bandwidth, but makes it hard to follow. I did honestly get confused about the difference between rp (the relying party name) and rp_id, where the challenge provides rp, and the browser response use rp_id.</p> <p>It can be easy to point fingers and say &quot;ohh William, you're just not reading it properly and are stupid&quot;. Am I? Or is it that humans find it really hard to parse data like this, and our brains are better suited to other tasks? Human factors are important to consider in specification design both in naming of values, consistency of their use, and appropriate communication as to how they are used properly. I'm finding this to be a barrier to correct implementation now (especially as the signature verification section is very fragmented and hard to follow ...).</p> <h2 id="crypto-steps-seem-complex-or-too-static">Crypto Steps seem complex or too static</h2> <p>There are a lot of possible choices here - there are 6 attestation formats and 5 attestation types. As some formats only do some types, there are then 11 verification paths you need to implement for all possible authenticators. I think this level of complexity will lead to mistakes over a large number of possible code branch paths, or lacking support for some device types which people may not have access to.</p> <p>I think it may have been better to limit the attestation format to one, well defined format, and within that to limit the attestation types available to suit a more broad range of uses.</p> <p>It feels a lot like these choice are part of some internal Google/MS/Other internal decisions for high security devices, or custom deviges, which will be internally used. It's leaked into the spec and it raises questions about the ability for people to meaningfully implement the full specification for all possible devices, let alone correctly.</p> <p>Some parts even omit details in a cryptographic operation, such as <a href="https://w3c.github.io/webauthn/#fido-u2f-attestation">here</a> in verification step 2, it doesn't even list what format the bytes are. (Hint: it's DER x509).</p> <h2 id="what-would-i-change">What would I change?</h2> <ul> <li>Be more specific</li> </ul> <p>There should be no assumptions about format types, what is in bytes. Be verbose, detailed and without ambiguity.</p> <ul> <li>Use type safe, length checked structures.</li> </ul> <p>I would probably make the entire thing a single CBOR structure which contains other nested structures as required. We should never have to hand-parse bytes in 2019, especially when there is a great deal of evidence to show the risks of expecting people to do this.</p> <ul> <li>Don't assume object orientation</li> </ul> <p>I think simpler, flatter structures in the json/cbor would have helped, and been clearer to implement, rather than the really complex maze of types currently involved.</p> <h2 id="summary">Summary</h2> <p>Despite these concerns, I still think webauthn is a really good standard, and I really do think it will become the future of authentication. I'm hoping to help make that a reality in opensource and I hope that in the future I can contribute to further development and promotion of webauthn.</p> The Case for Ethics in OpenSource Sun, 28 Apr 2019 00:00:00 +0000 Unknown https://fy.blackhats.net.au/blog/2019-04-28-the-case-for-ethics-in-opensource/ https://fy.blackhats.net.au/blog/2019-04-28-the-case-for-ethics-in-opensource/ <h1 id="the-case-for-ethics-in-opensource">The Case for Ethics in OpenSource</h1> <p>For a long time there have been incidents in technology which have caused negative effects on people - from leaks of private data, to interfaces that are not accessible, to even issues like UI's doing things that may try to subvert a persons intent. I'm sure there are many more: and we could be here all day listing the various issues that exist in technology, from small to great.</p> <p>The theme however is that these issues continue to happen: we continue to make decisions in applications that can have consequences to humans.</p> <p>Software is pointless without people. People create software, people deploy software, people interact with software, and even software indirectly can influence people's lives. At every layer people exist, and all software will affect them in some ways.</p> <p>I think that today, we have made a lot of progress in our communities around the deployment of code's of conduct. These are great, and really help us to discuss the decisions and actions we take within our communities - with the people who create the software. I would like this to go further, where we can have a framework to discuss the effect of software on people that we write: the people that deploy, interact with and are influenced by our work.</p> <h2 id="disclaimers">Disclaimers</h2> <p>I'm not a specialist in ethics or morality: I'm not a registered or certified engineer in the legal sense. Finally, like all humans I am a product of my experiences which causes all my view points to be biased through the lens of my experience.</p> <p>Additionally, I specialise in Identity Management software, so many of the ideas and issues I have encountered are really specific to this domain - which means I may overlook the issues in other areas. I also have a &quot;security&quot; mindset which also factors into my decisions too.</p> <p>Regardless I hope that this is a starting point to recieve further input and advice from others, and a place where we can begin to improve.</p> <h2 id="the-problem">The Problem</h2> <p>TODO: Discuss data handling practices</p> <p>Let's consider some issues and possible solutions in work that I'm familiar with - identity management software. Lets list a few &quot;features&quot;. (Please don't email me about how these are wrong, I know they are ...)</p> <ul> <li>Storing usernames as first and last name</li> <li>Storing passwords in cleartext.</li> <li>Deleting an account sets a flag to mark deletion</li> <li>Names are used as the primary key</li> <li>We request sex on signup</li> <li>To change account details, you have to use a command line tool</li> </ul> <p>Now &quot;technically&quot;, none of these decisions are incorrect at all. There is literally no bad technical decision here, and everything is &quot;technically correct&quot; (not always the best kind of correct).</p> <h2 id="what-do-we-want-to-achieve">What do we want to achieve?</h2> <p>There are lots of different issues here, but really want to prevent harm to a person. What is harm? Well that's a complex topic. To me, it could be emotional harm, disrespect of their person, it could be a feeling of a lack of control.</p> <p>I don't believe it's correct to dictate a set of rules that people should follow. People will be fatigued, and will find the process too hard. We need to trust that people can learn and want to improve. Instead I believe it's important we provide important points that people should be able to consider in a discussion around the development of software. The same way we discuss technical implementation details, we should discuss potential human impact in every change we have. To realise this, we need a short list of important factors that relate to humans.</p> <p>I think the following points are important to consider when designing software. These relate to general principles which I have learnt and researched.</p> <p>People should be respected to have:</p> <ul> <li>Informed consent</li> <li>Choice over how they are identified</li> <li>Ability to be forgotten</li> <li>Individual Autonomy</li> <li>Free from Harmful Discrimination</li> <li>Privacy</li> <li>Ability to meaningfully access and use software</li> </ul> <p>There is already some evidence in research papers to show that there are strong reasons for moral positions in software. For example, to prevent harm to come to people, to respect peoples autonomy and to conform to privacy legislation ( <a href="https://plato.stanford.edu/entries/it-privacy/#MorReaForProPerDat">source</a> ).</p> <h2 id="let-s-apply-these">Let's apply these</h2> <p>Given our set of &quot;features&quot;, lets now discuss these with the above points in mind.</p> <ul> <li>Storing usernames as first and last name</li> </ul> <p>This point clearly is in violation of the ability to choose how people are identified - some people may only have a single name, some may have multiple family names. On a different level this also violates the harmful discrimination rule due to the potential to disrespect individuals with cultures that have different name schemes compared to western/English societies.</p> <p>A better way to approach this is &quot;displayName&quot; as a freetext UTF8 case sensitive field, and to allow substring search over the content (rather than attempting to sort by first/last name which also has a stack of issues).</p> <ul> <li>Storing passwords in cleartext.</li> </ul> <p>This one is a violation of privacy, that we risk the exposure of a password which <em>may</em> have been reused (we can't really stop password reuse, we need to respect human behaviour). Not only that some people may assume we DO hash these correctly, so we actually are violating informed consent as we didn't disclose the method of how we store these details.</p> <p>A better thing here is to hash the password, or at least to disclose how it will be stored and used.</p> <ul> <li>Deleting an account sets a flag to mark deletion</li> </ul> <p>This violates the ability to be forgotten, because we aren't really deleting the account. It also breaks informed consent, because we are being &quot;deceptive&quot; about what our software is actually doing compared to the intent of the users request</p> <p>A better thing is to just delete the account, or if not possible, delete all user data and leave a tombstone inplace that represents &quot;an account was here, but no details associated&quot;.</p> <ul> <li>Names are used as the primary key</li> </ul> <p>This violates choice over identification, especially for women who have a divorce, or individuals who are transitioning or just people who want to change their name in general. The reason for the name change doesn't matter - what matters is we need to respect peoples right to identification.</p> <p>A better idea is to use UUID/ID numbers as a primary key, and have name able to be changed at any point in time.</p> <ul> <li>We request sex on signup</li> </ul> <p>Violates a privacy as a first point - we probably have no need for the data unless we are a medical application, so we should never ask for this at all. We also need to disclose why we need this data to satisfy informed consent, and potentially to allow them to opt-out of providing the data. Finally (if we really require this), to not violate self identification, we need to allow this to be a free-text field rather than a Male/Female boolean. This is not just in respect of individuals who are LGBTQI+, but the reality that there are biologically people who medically are neither. We also need to allow this to be changed at any time in the future. This in mind Sex and Gender are different concepts, so we should be careful which we request - Sex is the medical term of a person's genetics, and Gender is who the person identifies as.</p> <p>Not only this, because this is a very personal piece of information, we must disclose how we protect this information from access, who can see it, and if or how we'll ever share it with other systems or authorities.</p> <p>Generally, we probably don't need to know, so don't ask for it at all.</p> <ul> <li>To change account details, you have to use a command line tool</li> </ul> <p>This violates a users ability to meaningfully access and use software - remember, people come from many walks of life and all have different skill sets, but using command line tools is not something we can universally expect.</p> <p>A proper solution here is at minimum a web/graphical self management portal that is easy to access and follows proper UX/UI design rules, and for a business deploying, a service desk with humans involved that can support and help people change details on their account on their behalf if the person is unable to self-support via the web service.</p> <h2 id="proposal">Proposal</h2> <p>I think that OpenSource should aim to have a code of ethics - the same way we have a code of conduct to guide our behaviour internally to a project, we should have a framework to promote discussion of people's rights that use, interact with and are affected by our work. We should not focus on technical matters only, but should be promoting people at the core of all our work. Every decision we make is not just technical, but social.</p> <p>I'm sure that there are more points that could be considere than what I have listed here: I'd love to hear feedback to william at blackhats.net.au. Thanks!</p> Using Rust Generics to Enforce DB Record State Sat, 13 Apr 2019 00:00:00 +0000 Unknown https://fy.blackhats.net.au/blog/2019-04-13-using-rust-generics-to-enforce-db-record-state/ https://fy.blackhats.net.au/blog/2019-04-13-using-rust-generics-to-enforce-db-record-state/ <h1 id="using-rust-generics-to-enforce-db-record-state">Using Rust Generics to Enforce DB Record State</h1> <p>In a database, entries go through a lifecycle which represents what attributes they have have, db record keys, and if they have conformed to schema checking.</p> <p>I'm currently working on a (private in 2019, public in july 2019) project which is a NoSQL database writting in Rust. To help us manage the correctness and lifecycle of database entries, I have been using advice from the <a href="https://docs.rust-embedded.org/book/static-guarantees/state-machines.html">Rust Embedded Group's Book.</a></p> <p>As I have mentioned in the past, state machines are a great way to design code, so let's plot out the state machine we have for Entries:</p> <h2 id="entry-state-machine">Entry State Machine</h2> <p>The lifecyle is:</p> <ul> <li>A new entry is submitted by the user for creation</li> <li>We schema check that entry</li> <li>If it passes schema, we commit it and assign internal ID's</li> <li>When we search the entry, we retrieve it by internal ID's</li> <li>When we modify the entry, we need to recheck it's schema before we commit it back</li> <li>When we delete, we just remove the entry.</li> </ul> <p>This leads to a state machine of:</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>| </span><span>(create operation) </span><span>| </span><span>v </span><span>[ New + Invalid ] -(schema check)-&gt; [ New + Valid ] </span><span> | </span><span> (send to backend) </span><span> | </span><span> v v-------------\ </span><span>[Commited + Invalid] &lt;-(modify operation)- [ Commited + Valid ] | </span><span>| ^ \ (write to backend) </span><span>\--------------(schema check)-------------/ ---------------/ </span></code></pre> <p>This is a bit rough - The version on my whiteboard was better :)</p> <p>The main observation is that we are focused only on the commitability and validty of entries - not about where they are or if the commit was a success.</p> <h2 id="entry-structs">Entry Structs</h2> <p>So to make these states work we have the following structs:</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>struct EntryNew; </span><span>struct EntryCommited; </span><span> </span><span>struct EntryValid; </span><span>struct EntryInvalid; </span><span> </span><span>struct Entry&lt;STATE, VALID&gt; { </span><span> state: STATE, </span><span> valid: VALID, </span><span> // Other db junk goes here :) </span><span>} </span></code></pre> <p>We can then use these to establish the lifecycle with functions (similar) to this:</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>impl Entry&lt;EntryNew, EntryInvalid&gt; { </span><span> fn new() -&gt; Self { </span><span> Entry { </span><span> state: EntryNew, </span><span> valid: EntryInvalid, </span><span> ... </span><span> } </span><span> } </span><span> </span><span>} </span><span> </span><span>impl&lt;STATE&gt; Entry&lt;STATE, EntryInvalid&gt; { </span><span> fn validate(self, schema: Schema) -&gt; Result&lt;Entry&lt;STATE, EntryValid&gt;, ()&gt; { </span><span> if schema.check(self) { </span><span> Ok(Entry { </span><span> state: self.state, </span><span> valid: EntryValid, </span><span> ... </span><span> }) </span><span> } else { </span><span> Err(()) </span><span> } </span><span> } </span><span> </span><span> fn modify(&amp;mut self, ...) { </span><span> // Perform any modifications on the entry you like, only works </span><span> // on invalidated entries. </span><span> } </span><span>} </span><span> </span><span>impl&lt;STATE&gt; Entry&lt;STATE, EntryValid&gt; { </span><span> fn seal(self) -&gt; Entry&lt;EntryCommitted, EntryValid&gt; { </span><span> // Assign internal id&#39;s etc </span><span> Entry { </span><span> state: EntryCommited, </span><span> valid: EntryValid, </span><span> } </span><span> } </span><span> </span><span> fn compare(&amp;self, other: Entry&lt;STATE, EntryValid&gt;) -&gt; ... { </span><span> // Only allow compares on schema validated/normalised </span><span> // entries, so that checks don&#39;t have to be schema aware </span><span> // as the entries are already in a comparable state. </span><span> } </span><span>} </span><span> </span><span>impl Entry&lt;EntryCommited, EntryValid&gt; { </span><span> fn invalidate(self) -&gt; Entry&lt;EntryCommited, EntryInvalid&gt; { </span><span> // Invalidate an entry, to allow modifications to be performed </span><span> // note that modifications can only be applied once an entry is created! </span><span> Entry { </span><span> state: self.state, </span><span> valid: EntryInvalid, </span><span> } </span><span> } </span><span>} </span></code></pre> <p>What this allows us to do importantly is to control when we apply search terms, send entries to the backend for storage and more. Benefit is this is compile time checked, so you can never send an entry to a backend that is <em>not</em> schema checked, or run comparisons or searches on entries that aren't schema checked, and you can even only modify or delete something once it's created. For example other parts of the code now have:</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>impl BackendStorage { </span><span> // Can only create if no db id&#39;s are assigned, IE it must be new. </span><span> fn create(&amp;self, ..., entry: Entry&lt;EntryNew, EntryValid&gt;) -&gt; Result&lt;...&gt; { </span><span> } </span><span> </span><span> // Can only modify IF it has been created, and is validated. </span><span> fn modify(&amp;self, ..., entry: Entry&lt;EntryCommited, EntryValid&gt;) -&gt; Result&lt;...&gt; { </span><span> } </span><span> </span><span> // Can only delete IF it has been created and committed. </span><span> fn delete(&amp;self, ..., entry: Entry&lt;EntryCommited, EntryValid&gt;) -&gt; Result&lt;...&gt; { </span><span> } </span><span>} </span><span> </span><span>impl Filter&lt;STATE&gt; { </span><span> // Can only apply filters (searches) if the entry is schema checked. This has an </span><span> // important behaviour, where we can schema normalise. Consider a case-insensitive </span><span> // type, we can schema-normalise this on the entry, then our compare can simply </span><span> // be a string.compare, because we assert both entries *must* have been through </span><span> // the normalisation routines! </span><span> fn apply_filter(&amp;self, ..., entry: &amp;Entry&lt;STATE, EntryValid&gt;) -&gt; Result&lt;bool, ...&gt; { </span><span> } </span><span>} </span></code></pre> <h2 id="using-this-with-serde">Using this with Serde?</h2> <p>I have noticed that when we serialise the entry, that this causes the valid/state field to <em>not</em> be compiled away - because they <em>have</em> to be serialised, regardless of the empty content meaning the compiler can't eliminate them.</p> <p>A future cleanup will be to have a serialised DBEntry form such as the following:</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>struct DBEV1 { </span><span> // entry data here </span><span>} </span><span> </span><span>enum DBEntryVersion { </span><span> V1(DBEV1) </span><span>} </span><span> </span><span>struct DBEntry { </span><span> data: DBEntryVersion </span><span>} </span><span> </span><span>impl From&lt;Entry&lt;EntryNew, EntryValid&gt;&gt; for DBEntry { </span><span> fn from(e: Entry&lt;EntryNew, EntryValid&gt;) -&gt; Self { </span><span> // assign db id&#39;s, and return a serialisable entry. </span><span> } </span><span>} </span><span> </span><span>impl From&lt;Entry&lt;EntryCommited, EntryValid&gt;&gt; for DBEntry { </span><span> fn from(e: Entry&lt;EntryCommited, EntryValid&gt;) -&gt; Self { </span><span> // Just translate the entry to a serialisable form </span><span> } </span><span>} </span></code></pre> <p>This way we still have the zero-cost state on Entry, but we are able to move to a versioned seralised structure, and we minimise the run time cost.</p> <h2 id="testing-the-entry">Testing the Entry</h2> <p>To help with testing, I needed to be able to shortcut and move between anystate of the entry so I could quickly make fake entries, so I added some unsafe methods:</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>#[cfg(test)] </span><span>unsafe fn to_new_valid(self, Entry&lt;EntryNew, EntryInvalid&gt;) -&gt; { </span><span> Entry { </span><span> state: EntryNew, </span><span> valid: EntryValid </span><span> } </span><span>} </span></code></pre> <p>These allow me to setup and create small unit tests where I may not have a full backend or schema infrastructure, so I can test specific aspects of the entries and their lifecycle. It's limited to test runs only, and marked unsafe. It's not &quot;technically&quot; memory unsafe, but it's unsafe from the view of &quot;it could absolutely mess up your database consistency guarantees&quot; so you have to really want it.</p> <h2 id="summary">Summary</h2> <p>Using statemachines like this, really helped me to clean up my code, make stronger assertions about the correctness of what I was doing for entry lifecycles, and means that I have more faith when I and future-contributors will work on the code base that we'll have compile time checks to ensure we are doing the right thing - to prevent data corruption and inconsistency.</p> Debugging MacOS bluetooth audio stutter Mon, 08 Apr 2019 00:00:00 +0000 Unknown https://fy.blackhats.net.au/blog/2019-04-08-debugging-macos-bluetooth-audio-stutter/ https://fy.blackhats.net.au/blog/2019-04-08-debugging-macos-bluetooth-audio-stutter/ <h1 id="debugging-macos-bluetooth-audio-stutter">Debugging MacOS bluetooth audio stutter</h1> <p>I was noticing that audio to my bluetooth headphones from my iPhone was always flawless, but I started to noticed stutter and drops from my MBP. After exhausting some basic ideas, I was stumped.</p> <p>To the duck duck go machine, and I searched for issues with bluetooth known issues. Nothing appeared.</p> <p>However, I then decided to debug the issue - thankfully there was plenty of advice on this matter. Press shift + option while clicking bluetooth in the menu-bar, and then you have a debug menu. You can also open Console.app and search for &quot;bluetooth&quot; to see all the bluetooth related logs.</p> <p>I noticed that when the audio stutter occured that the following pattern was observed.</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>default 11:25:45.840532 +1000 wirelessproxd About to scan for type: 9 - rssi: -90 - payload: &lt;00000000 00000000 00000000 00000000 00000000 0000&gt; - mask: &lt;00000000 00000000 00000000 00000000 00000000 0000&gt; - peers: 0 </span><span>default 11:25:45.840878 +1000 wirelessproxd Scan options changed: YES </span><span>error 11:25:46.225839 +1000 bluetoothaudiod Error sending audio packet: 0xe00002e8 </span><span>error 11:25:46.225899 +1000 bluetoothaudiod Too many outstanding packets. Drop packet of 8 frames (total drops:451 total sent:60685 percentDropped:0.737700) Outstanding:17 </span></code></pre> <p>There was always a scan, just before the stutter initiated. So what was scanning?</p> <p>I searched for the error related to packets, and there were a lot of false leads. From weird apps to dodgy headphones. In this case I could eliminate both as the headphones worked with other devices, and I don't have many apps installed.</p> <p>So I went back and thought about what macOS services could be the problem, and I found that airdrop would scan periodically for other devices to send and recieve files. Disabling Airdrop from the sharing menu in System Prefrences cleared my audio right up.</p> <p>UPDATE 2019-12-20: It looks like the Airdrop sharing option in system preferences has been removed in 10.15, so I don't believe it's possible to resove this issue now - audio stutter forever!</p> GDB autoloads for 389 DS Wed, 03 Apr 2019 00:00:00 +0000 Unknown https://fy.blackhats.net.au/blog/2019-04-03-gdb-autoloads-for-389-ds/ https://fy.blackhats.net.au/blog/2019-04-03-gdb-autoloads-for-389-ds/ <h1 id="gdb-autoloads-for-389-ds">GDB autoloads for 389 DS</h1> <p>I've been writing a set of extensions to help debug 389-ds a bit easier. Thanks to the magic of python, writing GDB extensions is really easy.</p> <p>On OpenSUSE, when you start your DS instance under GDB, all of the extensions are automatically loaded. This will help make debugging a breeze.</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>zypper in 389-ds gdb </span><span>gdb /usr/sbin/ns-slapd </span><span> </span><span>GNU gdb (GDB; openSUSE Tumbleweed) 8.2 </span><span>(gdb) ds- </span><span>ds-access-log ds-backtrace </span><span>(gdb) set args -d 0 -D /etc/dirsrv/slapd-&lt;instance name&gt; </span><span>(gdb) run </span><span>... </span></code></pre> <p>All the extensions are under the ds- namespace, so they are easy to find. There are some new ones on the way, which I'll discuss here too:</p> <h2 id="ds-backtrace">ds-backtrace</h2> <p>As DS is a multithreaded process, it can be really hard to find the active thread involved in a problem. So we provided a command that knows how to fold duplicated stacks, and to highlight idle threads that you can (generally) skip over.</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>===== BEGIN ACTIVE THREADS ===== </span><span>Thread 37 (LWP 70054)) </span><span>Thread 36 (LWP 70053)) </span><span>Thread 35 (LWP 70052)) </span><span>Thread 34 (LWP 70051)) </span><span>Thread 33 (LWP 70050)) </span><span>Thread 32 (LWP 70049)) </span><span>Thread 31 (LWP 70048)) </span><span>Thread 30 (LWP 70047)) </span><span>Thread 29 (LWP 70046)) </span><span>Thread 28 (LWP 70045)) </span><span>Thread 27 (LWP 70044)) </span><span>Thread 26 (LWP 70043)) </span><span>Thread 25 (LWP 70042)) </span><span>Thread 24 (LWP 70041)) </span><span>Thread 23 (LWP 70040)) </span><span>Thread 22 (LWP 70039)) </span><span>Thread 21 (LWP 70038)) </span><span>Thread 20 (LWP 70037)) </span><span>Thread 19 (LWP 70036)) </span><span>Thread 18 (LWP 70035)) </span><span>Thread 17 (LWP 70034)) </span><span>Thread 16 (LWP 70033)) </span><span>Thread 15 (LWP 70032)) </span><span>Thread 14 (LWP 70031)) </span><span>Thread 13 (LWP 70030)) </span><span>Thread 12 (LWP 70029)) </span><span>Thread 11 (LWP 70028)) </span><span>Thread 10 (LWP 70027)) </span><span>#0 0x00007ffff65db03c in pthread_cond_wait@@GLIBC_2.3.2 () at /lib64/libpthread.so.0 </span><span>#1 0x00007ffff66318b0 in PR_WaitCondVar () at /usr/lib64/libnspr4.so </span><span>#2 0x00000000004220e0 in [IDLE THREAD] connection_wait_for_new_work (pb=0x608000498020, interval=4294967295) at /home/william/development/389ds/ds/ldap/servers/slapd/connection.c:970 </span><span>#3 0x0000000000425a31 in connection_threadmain () at /home/william/development/389ds/ds/ldap/servers/slapd/connection.c:1536 </span><span>#4 0x00007ffff6637484 in None () at /usr/lib64/libnspr4.so </span><span>#5 0x00007ffff65d4fab in start_thread () at /lib64/libpthread.so.0 </span><span>#6 0x00007ffff6afc6af in clone () at /lib64/libc.so.6 </span></code></pre> <p>This example shows that there are 17 idle threads (look at frame 2) here, that all share the same trace.</p> <h2 id="ds-access-log">ds-access-log</h2> <p>The access log is buffered before writing, so if you have a coredump, and want to see the last few events <em>before</em> they were written to disk, you can use this to display the content:</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>(gdb) ds-access-log </span><span>===== BEGIN ACCESS LOG ===== </span><span>$2 = 0x7ffff3c3f800 &quot;[03/Apr/2019:10:58:42.836246400 +1000] conn=1 fd=64 slot=64 connection from 127.0.0.1 to 127.0.0.1 </span><span>[03/Apr/2019:10:58:42.837199400 +1000] conn=1 op=0 BIND dn=\&quot;\&quot; method=128 version=3 </span><span>[03/Apr/2019:10:58:42.837694800 +1000] conn=1 op=0 RESULT err=0 tag=97 nentries=0 etime=0.0001200300 dn=\&quot;\&quot; </span><span>[03/Apr/2019:10:58:42.838881800 +1000] conn=1 op=1 SRCH base=\&quot;\&quot; scope=2 filter=\&quot;(objectClass=*)\&quot; attrs=ALL </span><span>[03/Apr/2019:10:58:42.839107600 +1000] conn=1 op=1 RESULT err=32 tag=101 nentries=0 etime=0.0001070800 </span><span>[03/Apr/2019:10:58:42.840687400 +1000] conn=1 op=2 UNBIND </span><span>[03/Apr/2019:10:58:42.840749500 +1000] conn=1 op=2 fd=64 closed - U1 </span><span>&quot;, &#39;\276&#39; &lt;repeats 3470 times&gt; </span></code></pre> <p>At the end the line that repeats shows the log is &quot;empty&quot; in that segment of the buffer.</p> <h2 id="ds-entry-print">ds-entry-print</h2> <p>This command shows the in-memory entry. It can be common to see Slapi_Entry * pointers in the codebase, so being able to display these is really helpful to isolate what's occuring on the entry. Your first argument should be the Slapi_Entry pointer.</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>(gdb) ds-entry-print ec </span><span>Display Slapi_Entry: cn=config </span><span>cn: config </span><span>objectClass: top </span><span>objectClass: extensibleObject </span><span>objectClass: nsslapdConfig </span><span>nsslapd-schemadir: /opt/dirsrv/etc/dirsrv/slapd-standalone1/schema </span><span>nsslapd-lockdir: /opt/dirsrv/var/lock/dirsrv/slapd-standalone1 </span><span>nsslapd-tmpdir: /tmp </span><span>nsslapd-certdir: /opt/dirsrv/etc/dirsrv/slapd-standalone1 </span><span>... </span></code></pre> Programming Lessons and Methods Tue, 26 Feb 2019 00:00:00 +0000 Unknown https://fy.blackhats.net.au/blog/2019-02-26-programming-lessons-and-methods/ https://fy.blackhats.net.au/blog/2019-02-26-programming-lessons-and-methods/ <h1 id="programming-lessons-and-methods">Programming Lessons and Methods</h1> <p>Everyone has their own lessons and methods that they use when they approaching programming. These are the lessons that I have learnt, which I think are the most important when it comes to design, testing and communication.</p> <h2 id="comments-and-design">Comments and Design</h2> <p>Programming is the art of writing human readable code, that a machine will eventually run. Your program needs to be reviewed, discussed and parsed by another human. That means you need to write your program in a way they can understand first.</p> <p>Rather than rushing into code, and hacking until it works, I find it's great to start with comments such as:</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>fn data_access(search: Search) -&gt; Type { </span><span> // First check the search is valid </span><span> // * No double terms </span><span> // * All schema is valid </span><span> </span><span> // Retrieve our data based on the search </span><span> </span><span> // if debug, do an un-indexed assert the search matches </span><span> </span><span> // Do any need transform </span><span> </span><span> // Return the data </span><span>} </span></code></pre> <p>After that, I walk away, think about the issue, come back, maybe tweak these comments. When I eventually fill in the code inbetween, I leave all the comments in place. This really helps my future self understand what I was thinking, but it also helps other people understand too.</p> <h2 id="state-machines">State Machines</h2> <p>State machines are a way to design and reason about the states a program can be in. They allow exhaustive represenations of all possible outcomes of a function. A simple example is a microwave door.</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>/----\ /----- close ----\ /-----\ </span><span>| \ / v v | </span><span>| ------------- --------------- | </span><span>open | Door Open | | Door Closed | close </span><span>| ------------- --------------- | </span><span>| ^ ^ / \ | </span><span>\---/ \------ open ----/ \----/ </span></code></pre> <p>When the door is open, opening it again does nothing. Only when the door is open, and we close the door (and event), does the door close (a transition). Once closed, the door can not be closed any more (event does nothing). It's when we open the door now, that a state change can occur.</p> <p>There is much more to state machines than this, but they allow us as humans to reason about our designs and model our programs to have all possible outcomes considered.</p> <h2 id="zero-one-and-infinite">Zero, One and Infinite</h2> <p>In mathematics there are only three numbers that matter. Zero, One and Infinite. It turns out the same is true in a computer too.</p> <p>When we are making a function, we can define limits in these terms. For example:</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>fn thing(argument: Type) </span></code></pre> <p>In this case, argument is &quot;One&quot; thing, and must be one thing.</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>fn thing(argument: Option&lt;Type&gt;) </span></code></pre> <p>Now we have argument as an option, so it's &quot;Zero&quot; or &quot;One&quot;.</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>fn thing(argument: Vec&lt;Type&gt;) </span></code></pre> <p>Now we have argument as vec (array), so it's &quot;Zero&quot; to &quot;Infinite&quot;.</p> <p>When we think about this, our functions have to handle these cases properly. We don't write functions that take a vec with only two items, we write a function with two arguments where each one must exist. It's hard to handle &quot;two&quot; - it's easy to handle two cases of &quot;one&quot;.</p> <p>It also is a good guide for how to handle data sets, assuming they could always be infinite in size (or at least any arbitrary size).</p> <p>You can then apply this to tests. In a test given a function of:</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>fn test_me(a: Option&lt;Type&gt;, b: Vec&lt;Type&gt;) </span></code></pre> <p>We know we need to test permutations of:</p> <ul> <li>a is &quot;Zero&quot; or &quot;One&quot; (Some, None)</li> <li>b is &quot;Zero&quot;, &quot;One&quot; or &quot;Infinite&quot; (.len() == 0, .len() == 1, .len() &gt; 0)</li> </ul> <p>Note: Most languages don't have an array type that is &quot;One to Infinite&quot;, IE non-empty. If you want this condition (at least one item), you have to assert it yourself ontop of the type system.</p> <h2 id="correct-simple-fast">Correct, Simple, Fast</h2> <p>Finally, we can put all these above tools together and apply a general philosophy. When writing a program, first make it correct, then simpify the program, then make it fast.</p> <p>If you don't do it in this order you will hit barriers - social and technical. For example, if you make something fast, simple, correct, you will likely have issues that can be fixed without making a decrease in performance. People don't like it when you introduce a patch that drops performance, so as a result correctness is now sacrificed. (Spectre anyone?)</p> <p>If you make something too simple, you may never be able to make it correctly handle all cases that exist in your application - likely facilitating a future rewrite to make it correct.</p> <p>If you do correct, fast, simple, then your program will be correct, and fast, but hard for a human to understand. Because programming is the art of communicating intent to a person sacrificing simplicity in favour of fast will make it hard to involve new people and educate and mentor them into development of your project.</p> <ul> <li>Correct: Does it behave correctly, handle all states and inputs correctly?</li> <li>Simple: Is it easy to comprehend and follow for a human reader?</li> <li>Fast: Is it performant?</li> </ul> Meaningful 2fa on modern linux Tue, 12 Feb 2019 00:00:00 +0000 Unknown https://fy.blackhats.net.au/blog/2019-02-12-meaningful-2fa-on-modern-linux/ https://fy.blackhats.net.au/blog/2019-02-12-meaningful-2fa-on-modern-linux/ <h1 id="meaningful-2fa-on-modern-linux">Meaningful 2fa on modern linux</h1> <p>Recently I heard of someone asking the question:</p> <p>&quot;I have an AD environment connected with &lt;product&gt; IDM. I want to have 2fa/mfa to my linux machines for ssh, that works when the central servers are offline. What's the best way to achieve this?&quot;</p> <p>Today I'm going to break this down - but the conclusion for the lazy is:</p> <p><em>This is not realistically possible today: use ssh keys with ldap distribution, and mfa on the workstations, with full disk encryption</em>.</p> <h2 id="background">Background</h2> <p>So there are a few parts here. AD is for intents and purposes an LDAP server. The &lt;product&gt; is also an LDAP server, that syncs to AD. We don't care if that's 389-ds, freeipa or vendor solution. The results are basically the same.</p> <p>Now the linux auth stack is, and will always use pam for the authentication, and nsswitch for user id lookups. Today, we assume that most people run sssd, but pam modules for different options are possible.</p> <p>There are a stack of possible options, and they all have various flaws.</p> <ul> <li>FreeIPA + 2fa</li> <li>PAM TOTP modules</li> <li>PAM radius to a TOTP server</li> <li>Smartcards</li> </ul> <h2 id="freeipa-2fa">FreeIPA + 2fa</h2> <p>Now this is the one most IDM people would throw out. The issue here is the person already has AD and a vendor product. They don't need a third solution.</p> <p>Next is the fact that FreeIPA stores the TOTP in the LDAP, which means FreeIPA has to be online for it to work. So this is eliminated by the &quot;central servers offline&quot; requirement.</p> <h2 id="pam-radius-to-totp-server">PAM radius to TOTP server</h2> <p>Same as above: An extra product, and you have a source of truth that can go down.</p> <h2 id="pam-totp-module-on-hosts">PAM TOTP module on hosts</h2> <p>Okay, even if you can get this to scale, you need to send the private seed material of every TOTP device that could login to the machine, to every machine. That means <em>any</em> compromise, compromises every TOTP token on your network. Bad place to be in.</p> <h2 id="smartcards">Smartcards</h2> <p>Are notoriously difficult to have functional, let alone with SSH. Don't bother. (Where the Smartcard does TLS auth to the SSH server this is.)</p> <h2 id="come-on-william-why-are-you-so-doom-and-gloom">Come on William, why are you so doom and gloom!</h2> <p>Lets back up for a second and think about what we we are trying to prevent by having mfa at all. We want to prevent single factor compromise from having a large impact <em>and</em> we want to prevent brute force attacks. (There are probably more reasons, but these are the ones I'll focus on).</p> <p>So the best answer: Use mfa on the workstation (password + totp), then use ssh keys to the hosts.</p> <p>This means the target of the attack is small, and the workstation can be protected by things like full disk encryption and group policy. To sudo on the host you still need the password. This makes sudo MFA to root as you need something know, and something you have.</p> <p>If you are extra conscious you can put your ssh keys on smartcards. This works on linux and osx workstations with yubikeys as I am aware. Apparently you can have ssh keys in TPM, which would give you tighter hardware binding, but I don't know how to achieve this (yet).</p> <p>To make all this better, you can distributed your ssh public keys in ldap, which means you gain the benefits of LDAP account locking/revocation, you can remove the keys instantly if they are breached, and you have very little admin overhead to configuration of this service on the linux server side. Think about how easy onboarding is if you only need to put your ssh key in one place and it works on every server! Let alone shutting down a compromised account: lock it in one place, and they are denied access to every server.</p> <p>SSSD as the LDAP client on the server can also cache the passwords (hashed) and the ssh public keys, which means a disconnected client will still be able to be authenticated to.</p> <p>At this point, because you have ssh key auth working, you could even <em>deny</em> password auth as an option in ssh altogether, eliminating an entire class of bruteforce vectors.</p> <p>For bonus marks: You can use AD as the generic LDAP server that stores your SSH keys. No additional vendor products needed, you already have everything required today, for free. Everyone loves free.</p> <h2 id="conclusion">Conclusion</h2> <p>If you want strong, offline capable, distributed mfa on linux servers, the only choice today is LDAP with SSH key distribution.</p> <p>Want to know more? This blog contains how-tos on SSH key distribution for AD, SSH keys on smartcards, and how to configure SSSD to use SSH keys from LDAP.</p> Using the latest 389-ds on OpenSUSE Wed, 30 Jan 2019 00:00:00 +0000 Unknown https://fy.blackhats.net.au/blog/2019-01-30-using-the-latest-389-ds-on-opensuse/ https://fy.blackhats.net.au/blog/2019-01-30-using-the-latest-389-ds-on-opensuse/ <h1 id="using-the-latest-389-ds-on-opensuse">Using the latest 389-ds on OpenSUSE</h1> <p>Thanks to some help from my friend who works on OBS, I've finally got a good package in review for submission to tumbleweed. However, if you are impatient and want to use the &quot;latest&quot; and greatest 389-ds version on OpenSUSE.</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>zypper ar obs://network:ldap network:ldap </span><span>zypper in 389-ds </span></code></pre> <h2 id="docker">Docker</h2> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>docker run --rm -i -t registry.opensuse.org/home/firstyear/containers/389-ds-container:latest </span></code></pre> <p>To make it persistent:</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>docker run -v 389ds_data:/data &lt;your options here ...&gt; registry.opensuse.org/home/firstyear/containers/389-ds-container:latest </span></code></pre> <p>Then to run the admin tools:</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>docker exec -i -t &lt;container name&gt; /usr/sbin/dsconf ... </span><span>docker exec -i -t &lt;container name&gt; /usr/sbin/dsidm ... </span></code></pre> <h2 id="testing-in-docker">Testing in docker?</h2> <p>If you are &quot;testing&quot; in docker (please don't do this in production: for production see above) you'll need to do some tweaks to get around the lack of systemd.</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>docker run -i -t opensuse/tumbleweed:latest </span><span>zypper ar obs://network:ldap network:ldap </span><span>zypper in 389-ds </span><span> </span><span>vim /usr/share/dirsrv/inf/defaults.inf </span><span># change the following to match: </span><span>with_systemd = 0 </span></code></pre> <h2 id="what-next">What next?</h2> <p>After this, you should now be able to follow our <a href="http://www.port389.org/docs/389ds/howto/quickstart.html">new quickstart guide</a> on the 389-ds website.</p> <p>If you followed the docker steps, skip to <a href="http://www.port389.org/docs/389ds/howto/quickstart.html#add-users-and-groups">adding users and groups</a></p> <p>The network:ldap repo and the container listed are updated when upstream makes releases so you'll always get the latest 389-ds</p> <p>EDIT: Updated 2019-04-03 to change repo as changes have progressed forward.</p> <p>EDIT: Updated 2019-08-27 Improve clarity about when you need to do docker tweaks, and add docker image steps</p> Structuring Rust Transactions Sat, 19 Jan 2019 00:00:00 +0000 Unknown https://fy.blackhats.net.au/blog/2019-01-19-structuring-rust-transactions/ https://fy.blackhats.net.au/blog/2019-01-19-structuring-rust-transactions/ <h1 id="structuring-rust-transactions">Structuring Rust Transactions</h1> <p>I've been working on a database-related project in Rust recently, which takes advantage of my <a href="https://crates.io/crates/concread">concurrently readable datastructures.</a> However I ran into a problem of how to structure Read/Write transaction structures that shared the reader code, and container multiple inner read/write types.</p> <h2 id="some-constraints">Some Constraints</h2> <p>To be clear, there are some constraints. A &quot;parent&quot; write, will only ever contain write transaction guards, and a read will only ever contain read transaction guards. This means we aren't going to hit any deadlocks in the code. Rust can't protect us from mis-ording locks. An additional requirement is that readers and a single write must be able to proceed simultaneously - but having a rwlock style writer or readers behaviour would still work here.</p> <h2 id="some-background">Some Background</h2> <p>To simplify this, imagine we have two concurrently readable datastructures. We'll call them db_a and db_b.</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>struct db_a { ... } </span><span> </span><span>struct db_b { ... } </span></code></pre> <p>Now, each of db_a and db_b has their own way to protect their inner content, but they'll return a DBWriteGuard or DBReadGuard when we call db_a.read()/write() respectively.</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>impl db_a { </span><span> pub fn read(&amp;self) -&gt; DBReadGuard { </span><span> ... </span><span> } </span><span> </span><span> pub fn write(&amp;self) -&gt; DBWriteGuard { </span><span> ... </span><span> } </span><span>} </span></code></pre> <p>Now we make a &quot;parent&quot; wrapper transaction such as:</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>struct server { </span><span> a: db_a, </span><span> b: db_b, </span><span>} </span><span> </span><span>struct server_read { </span><span> a: DBReadGuard, </span><span> b: DBReadGuard, </span><span>} </span><span> </span><span>struct server_write { </span><span> a: DBWriteGuard, </span><span> b: DBWriteGuard, </span><span>} </span><span> </span><span>impl server { </span><span> pub fn read(&amp;self) -&gt; server_read { </span><span> server_read { </span><span> self.a.read(), </span><span> self.b.read(), </span><span> } </span><span> } </span><span> </span><span> pub fn write(&amp;self) -&gt; server_write { </span><span> server_read { </span><span> self.a.write(), </span><span> self.b.write(), </span><span> } </span><span> } </span><span>} </span></code></pre> <h2 id="the-problem">The Problem</h2> <p>Now the problem is that on my server_read and server_write I want to implement a function for &quot;search&quot; that uses the same code. Search or a read or write should behave identically! I wanted to also avoid the use of macros as the can hide issues while stepping in a debugger like LLDB/GDB.</p> <p>Often the answer with rust is &quot;traits&quot;, to create an interface that types adhere to. Rust also allows default trait implementations, which sounds like it could be a solution here.</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>pub trait server_read_trait { </span><span> fn search(&amp;self) -&gt; SomeResult { </span><span> let result_a = self.a.search(...); </span><span> let result_b = self.b.search(...); </span><span> SomeResult(result_a, result_b) </span><span> } </span><span>} </span></code></pre> <p>In this case, the issue is that &amp;self in a trait is not aware of the fields in the struct - traits don't define that fields <em>must</em> exist, so the compiler can't assume they exist at all.</p> <p>Second, the type of self.a/b is unknown to the trait - because in a read it's a &quot;a: DBReadGuard&quot;, and for a write it's &quot;a: DBWriteGuard&quot;.</p> <p>The first problem can be solved by using a get_field type in the trait. Rust will also compile this out as an inline, so the <em>correct</em> thing for the type system is also the <em>optimal</em> thing at run time. So we'll update this to:</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>pub trait server_read_trait { </span><span> fn get_a(&amp;self) -&gt; ???; </span><span> </span><span> fn get_b(&amp;self) -&gt; ???; </span><span> </span><span> fn search(&amp;self) -&gt; SomeResult { </span><span> let result_a = self.get_a().search(...); // note the change from self.a to self.get_a() </span><span> let result_b = self.get_b().search(...); </span><span> SomeResult(result_a, result_b) </span><span> } </span><span>} </span><span> </span><span>impl server_read_trait for server_read { </span><span> fn get_a(&amp;self) -&gt; &amp;DBReadGuard { </span><span> &amp;self.a </span><span> } </span><span> // get_b is similar, so ommitted </span><span>} </span><span> </span><span>impl server_read_trait for server_write { </span><span> fn get_a(&amp;self) -&gt; &amp;DBWriteGuard { </span><span> &amp;self.a </span><span> } </span><span> // get_b is similar, so ommitted </span><span>} </span></code></pre> <p>So now we have the second problem remaining: for the server_write we have DBWriteGuard, and read we have a DBReadGuard. There was a much longer experimentation process, but eventually the answer was simpler than I was expecting. Rust allows traits to have Self types that enforce trait bounds rather than a concrete type.</p> <p>So provided that DBReadGuard and DBWriteGuard both implement &quot;DBReadTrait&quot;, then we can have the server_read_trait have a self type that enforces this. It looks something like:</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>pub trait DBReadTrait { </span><span> fn search(&amp;self) -&gt; ...; </span><span>} </span><span> </span><span>impl DBReadTrait for DBReadGuard { </span><span> fn search(&amp;self) -&gt; ... { ... } </span><span>} </span><span> </span><span>impl DBReadTrait for DBWriteGuard { </span><span> fn search(&amp;self) -&gt; ... { ... } </span><span>} </span><span> </span><span>pub trait server_read_trait { </span><span> type GuardType: DBReadTrait; // Say that GuardType must implement DBReadTrait </span><span> </span><span> fn get_a(&amp;self) -&gt; &amp;Self::GuardType; // implementors must return that type implementing the trait. </span><span> </span><span> fn get_b(&amp;self) -&gt; &amp;Self::GuardType; </span><span> </span><span> fn search(&amp;self) -&gt; SomeResult { </span><span> let result_a = self.get_a().search(...); </span><span> let result_b = self.get_b().search(...); </span><span> SomeResult(result_a, result_b) </span><span> } </span><span>} </span><span> </span><span>impl server_read_trait for server_read { </span><span> fn get_a(&amp;self) -&gt; &amp;DBReadGuard { </span><span> &amp;self.a </span><span> } </span><span> // get_b is similar, so ommitted </span><span>} </span><span> </span><span>impl server_read_trait for server_write { </span><span> fn get_a(&amp;self) -&gt; &amp;DBWriteGuard { </span><span> &amp;self.a </span><span> } </span><span> // get_b is similar, so ommitted </span><span>} </span></code></pre> <p>This works! We now have a way to write a single &quot;search&quot; type for our server read and write types. In my case, the DBReadTrait also uses a similar technique to define a search type shared between the DBReadGuard and DBWriteGuard.</p> SUSE Open Build Service cheat sheet Sat, 19 Jan 2019 00:00:00 +0000 Unknown https://fy.blackhats.net.au/blog/2019-01-19-suse-open-build-system-cheat-sheet/ https://fy.blackhats.net.au/blog/2019-01-19-suse-open-build-system-cheat-sheet/ <h1 id="suse-open-build-service-cheat-sheet">SUSE Open Build Service cheat sheet</h1> <p>Part of starting at SUSE has meant that I get to learn about Open Build Service. I've known that the project existed for a long time but I have never had a chance to use it. So far I'm thoroughly impressed by how it works and the features it offers.</p> <h2 id="as-a-consumer">As A Consumer</h2> <p>The best part of OBS is that it's trivial on OpenSUSE to consume content from it. Zypper can add projects with the command:</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>zypper ar obs://&lt;project name&gt; &lt;repo nickname&gt; </span><span>zypper ar obs://network:ldap network:ldap </span></code></pre> <p>I like to give the repo nickname (your choice) to be the same as the project name so I know what I have enabled. Once you run this you can easily consume content from OBS.</p> <h2 id="package-management">Package Management</h2> <p>As someone who has started to contribute to the suse 389-ds package, I've been slowly learning how this work flow works. OBS similar to GitHub/Lab allows a branching and request model.</p> <p>On OpenSUSE you will want to use the osc tool for your workflow:</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>zypper in osc </span><span># If you plan to use the &quot;service&quot; command </span><span>zypper in obs-service-tar obs-service-obs_scm obs-service-recompress obs-service-set_version obs-service-download_files python-xml obs-service-format_spec_file </span></code></pre> <p>You can branch from an existing project to make changes with:</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>osc branch &lt;project&gt; &lt;package&gt; </span><span>osc branch network:ldap 389-ds </span></code></pre> <p>This will branch the project to my home namespace. For me this will land in &quot;home:firstyear:branches:network:ldap&quot;. Now I can checkout the content on to my machine to work on it.</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>osc co &lt;project&gt; </span><span>osc co home:firstyear:branches:network:ldap </span></code></pre> <p>This will create the folder &quot;home:...:ldap&quot; in the current working directory.</p> <p>From here you can now work on the project. Some useful commands are:</p> <p>Add new files to the project (patches, new source tarballs etc).</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>osc add &lt;path to file&gt; </span><span>osc add feature.patch </span><span>osc add new-source.tar.xz </span></code></pre> <p>Edit the change log of the project (I think this is used in release notes?)</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>osc vc </span></code></pre> <p>To ammend your changes, use:</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>osc vc -e </span></code></pre> <p>Build your changes locally matching the system you are on. Packages normally build on all/most OpenSUSE versions and architectures, this will build just for your local system and arch.</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>osc build </span></code></pre> <p>Make sure you clean up files you aren't using any more with:</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>osc rm &lt;filename&gt; </span><span># This commands removes anything untracked by osc. </span><span>osc clean </span></code></pre> <p>Commit your changes to the OBS server, where a complete build will be triggered:</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>osc commit </span></code></pre> <p>View the results of the last commit:</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>osc results </span></code></pre> <p>Enable people to use your branch/project as a repository. You edit the project metadata and enable repo publishing:</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>osc meta prj -e &lt;name of project&gt; </span><span>osc meta prj -e home:firstyear:branches:network:ldap </span><span> </span><span># When your editor opens, change this section to enabled (disabled by default): </span><span>&lt;publish&gt; </span><span> &lt;enabled /&gt; </span><span>&lt;/publish&gt; </span></code></pre> <p>NOTE: In some cases if you have the package already installed, and you add the repo/update it won't install from your repo. This is because in SUSE packages have a notion of &quot;vendoring&quot;. They continue to update from the same repo as they were originally installed from. So if you want to change this you use:</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>zypper [d]up --from &lt;repo name&gt; </span></code></pre> <p>You can then create a &quot;request&quot; to merge your branch changes back to the project origin. This is:</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>osc sr </span></code></pre> <p>A helpful maintainer will then review your changes. You can see this with.</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>osc rq show &lt;your request id&gt; </span></code></pre> <p>If you change your request, to submit again, use:</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>osc sr </span></code></pre> <p>And it will ask if you want to replace (supercede) the previous request.</p> <p>I was also helped by a friend to provie a &quot;service&quot; configuration that allows generation of tar balls from git. It's not always appropriate to use this, but if the repo has a &quot;_service&quot; file, you can regenerate the tar with:</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>osc service ra </span></code></pre> <p>So far this is as far as I have gotten with OBS, but I already appreciate how great this work flow is for package maintainers, reviewers and consumers. It's a pleasure to work with software this well built.</p> <p>As an additional piece of information, it's a good idea to read the <a href="https://en.opensuse.org/openSUSE:Packaging_Patches_guidelines">OBS Packaging Guidelines</a></p> <p>: to be sure that you are doing the right thing!</p> <p># Acts as tail</p> <p>: osc bl osc r -v</p> <p># How to access the meta and docker stuff</p> <blockquote> <p>osc meta pkg -e osc meta prj -e</p> </blockquote> <p>(will add vim to the buildroot), then you can chroot (allow editing in the build root)</p> <blockquote> <p>osc chroot osc build -x vim</p> </blockquote> <p>-k &lt;dir&gt; keeps artifacts in directory dir IE rpm outputs</p> <p>oscrc buildroot variable, mount tmpfs to that location.</p> <p>docker privs SYS_ADMIN, SYS_CHROOT</p> <p>Multiple spec: commit second spec to pkg then:</p> <blockquote> <p>osc linkpac prj pkg prj new-link-pkg</p> </blockquote> <p>rpm --eval '%{variable}'</p> <p>%setup -n &quot;name of what the directory unpacks to, not what to rename to&quot;</p> <p>When no link exists</p> <blockquote> <p>osc submitpac destprj deskpkg</p> </blockquote> The idea of CI and Engineering Wed, 02 Jan 2019 00:00:00 +0000 Unknown https://fy.blackhats.net.au/blog/2019-01-02-the-idea-of-ci-and-engineering/ https://fy.blackhats.net.au/blog/2019-01-02-the-idea-of-ci-and-engineering/ <h1 id="the-idea-of-ci-and-engineering">The idea of CI and Engineering</h1> <p>In software development I see and interesting trend and push towards continuous integration, continually testing, and testing in production. These techniques are designed to allow faster feedback on errors, use real data for application testing, and to deliver features and changes faster.</p> <p>But is that really how people use software on devices? When we consider an operation like google or amazon, this always online technique may work, but what happens when we apply a continous integration and &quot;we'll patch it later&quot; mindset to devices like phones or internet of things?</p> <h2 id="what-happens-in-other-disciplines">What happens in other disciplines?</h2> <p>In real engineering disciplines like aviation or construction, techniques like this don't really work. We don't continually build bridges, then fix them when they break or collapse. There are people who provide formal analysis of materials, their characteristics. Engineers consider careful designs, constraints, loads and situations that may occur. The structure is planned, reviewed and verified mathematically. Procedures and oversight is applied to ensure correct building of the structure. Lessons are learnt from past failures and incidents and are applied into every layer of the design and construction process. Communication between engineers and many other people is critical to the process. Concerns are always addressed and managed.</p> <p>The first thing to note is that if we just built lots of scale-model bridges and continually broke them until we found their limits, this would waste many resources to do this. Bridges are carefully planned and proven.</p> <h2 id="so-whats-the-point-with-software">So whats the point with software?</h2> <p>Today we still have a mindset that continually breaking and building is a reasonable path to follow. It's not! It means that the only way to achieve quality is to have a large test suite (requires people and time to write), which has to be further derived from failures (and those failures can negatively affect real people), then we have to apply large amounts of electrical energy to continually run the tests. The test suites can't even guarantee complete coverage of all situations and occurances!</p> <p>This puts CI techniques out of reach of many application developers due to time and energy (translated to dollars) limits. Services like travis on github certainly helps to lower the energy requirement, but it doesn't stop the time and test writing requirements.</p> <p>No matter how many tests we have for a program, if that program is written in C or something else, we continually see faults and security/stability issues in that software.</p> <h2 id="what-if-we-ci-on-a-phone">What if we CI on ... a phone?</h2> <p>Today we even have hardware devices that are approached as though they &quot;test in production&quot; is a reasonable thing. It's not! People don't patch, telcos don't allow updates out to users, and those that are aware, have to do custom rom deployment. This creates an odd dichomtemy of &quot;haves&quot; and &quot;haves not&quot;, of those in technical know how who have a better experience, and the &quot;haves not&quot; who have to suffer potentially insecure devices. This is especially terrifying given how deeply personal phones are.</p> <p>This is a reality of our world. People do not patch. They do not patch phones, laptops, network devices and more. Even enterprises will avoid patching if possible. Rather than trying to shift the entire culture of humans to &quot;update always&quot;, we need to write software that can cope in harsh conditions, for long term. We only need to look to software in aviation to see we can absolutely achieve this!</p> <h2 id="what-should-we-do">What should we do?</h2> <p>I believe that for software developers to properly become software engineers we should look to engineers in civil and aviation industries. We need to apply:</p> <ul> <li>Regualation and ethics (Safety of people is always first)</li> <li>Formal verification</li> <li>Consider all software will run long term (5+ years)</li> <li>Improve team work and collaboration on designs and development</li> </ul> <p>The reality of our world is people are deploying devices (routers, networks, phones, lights, laptops more ...) where they may never be updated or patched in their service life. Even I'm guilty (I have a modem that's been unpatched for about 6 years but it's pretty locked down ...). As a result we need to rely on proof that the device <em>can not</em> fail at build time, rather than <em>patch it later</em> which may never occur! Putting formal verification first, and always considering user safety and rights first, shifts a large burden to us in terms of time. But many tools (Coq, fstar, rust ...) all make formal verification more accessible to use in our industry. Verifying our software is a far stronger assertion of quality than &quot;throw tests at it and hope it works&quot;.</p> <h2 id="you-re-crazy-william-and-also-wrong">You're crazy William, and also wrong</h2> <p>Am I? Looking at &quot;critical&quot; systems like iPhone encryption hardware, they are running the formally verified Sel4. We also heard at Kiwicon in 2018 that Microsoft and XBox are using formal verification to design their low levels of their system to prevent exploits from occuring in the first place.</p> <p>Over time our industry will evolve, and it will become easier and more cost effective to formally verify than to operate and deploy CI. This doesn't mean we don't need tests - it means that the first line of quality should be in verification of correctness using formal techniques rather than using tests and CI to prove correct behaviour. Tests are certainly still required to assert further behavioural elements of software.</p> <p>Today, if you want to do this, you should be looking at Coq and program extraction, fstar and the kremlin (project everest, a formally verified https stack), Rust (which has a subset of the safe language formally proven). I'm sure there are more, but these are the ones I know off the top of my head.</p> <h2 id="conclusion">Conclusion</h2> <p>Over time our industry <em>must</em> evolve to put the safety of humans first. To achive this we must look to other safety driven cultures such as aviation and civil engineering. Only by learning from their strict disciplines and behaviours can we start to provide software that matches behavioural and quality expectations humans have for software.</p> Useful USG pro 4 commands and hints Wed, 02 Jan 2019 00:00:00 +0000 Unknown https://fy.blackhats.net.au/blog/2019-01-02-useful-usg-pro-4-commands-and-hints/ https://fy.blackhats.net.au/blog/2019-01-02-useful-usg-pro-4-commands-and-hints/ <h1 id="useful-usg-pro-4-commands-and-hints">Useful USG pro 4 commands and hints</h1> <p>I've recently changed from a FreeBSD vm as my router to a Ubiquiti PRO USG4. It's a solid device, with many great features, and I'm really impressed at how it &quot;just works&quot; in many cases. So far my only disappointment is lack of documentation about the CLI, especially for debugging and auditing what is occuring in the system, and for troubleshooting steps. This post will aggregate some of my knowledge about the topic.</p> <h2 id="current-config">Current config</h2> <p>Show the current config with:</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>mca-ctrl -t dump-cfg </span></code></pre> <p>You can show system status with the &quot;show&quot; command. Pressing ? will cause the current compeletion options to be displayed. For example:</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span># show &lt;?&gt; </span><span>arp date dhcpv6-pd hardware </span></code></pre> <h2 id="dns">DNS</h2> <p>The following commands show the DNS statistics, the DNS configuration, and allow changing the cache-size. The cache-size is measured in number of records cached, rather than KB/MB. To make this permanent, you need to apply the change to config.json in your controllers sites folder.</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>show dns forwarding statistics </span><span>show system name-server </span><span>set service dns forwarding cache-size 10000 </span><span>clear dns forwarding cache </span></code></pre> <h2 id="logging">Logging</h2> <p>You can see and aggregate of system logs with</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>show log </span></code></pre> <p>Note that when you set firewall rules to &quot;log on block&quot; they go to dmesg, not syslog, so as a result you need to check dmesg for these.</p> <p>It's a great idea to forward your logs in the controller to a syslog server as this allows you to aggregate and see all the events occuring in a single time series (great when I was diagnosing an issue recently).</p> <h2 id="interfaces">Interfaces</h2> <p>To show the system interfaces</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>show interfaces </span></code></pre> <p>To restart your pppoe dhcp6c:</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>release dhcpv6-pd interface pppoe0 </span><span>renew dhcpv6-pd interface pppoe0 </span></code></pre> <p>There is a current issue where the firmware will start dhcp6c on eth2 and pppoe0, but the session on eth2 blocks the pppoe0 client. As a result, you need to release on eth2, then renew of pppoe0</p> <p>If you are using a dynamic prefix rather than static, you may need to reset your dhcp6c duid.</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>delete dhcpv6-pd duid </span></code></pre> <p>To restart an interface with the vyatta tools:</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>disconnect interface pppoe </span><span>connect interface pppoe </span></code></pre> <h2 id="openvpn">OpenVPN</h2> <p>I have setup customised OpenVPN tunnels. To show these:</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>show interfaces openvpn detail </span></code></pre> <p>These are configured in config.json with:</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span># Section: config.json - interfaces - openvpn </span><span> &quot;vtun0&quot;: { </span><span> &quot;encryption&quot;: &quot;aes256&quot;, </span><span> # This assigns the interface to the firewall zone relevant. </span><span> &quot;firewall&quot;: { </span><span> &quot;in&quot;: { </span><span> &quot;ipv6-name&quot;: &quot;LANv6_IN&quot;, </span><span> &quot;name&quot;: &quot;LAN_IN&quot; </span><span> }, </span><span> &quot;local&quot;: { </span><span> &quot;ipv6-name&quot;: &quot;LANv6_LOCAL&quot;, </span><span> &quot;name&quot;: &quot;LAN_LOCAL&quot; </span><span> }, </span><span> &quot;out&quot;: { </span><span> &quot;ipv6-name&quot;: &quot;LANv6_OUT&quot;, </span><span> &quot;name&quot;: &quot;LAN_OUT&quot; </span><span> } </span><span> }, </span><span> &quot;mode&quot;: &quot;server&quot;, </span><span> # By default, ubnt adds a number of parameters to the CLI, which </span><span> # you can see with ps | grep openvpn </span><span> &quot;openvpn-option&quot;: [ </span><span> # If you are making site to site tunnels, you need the ccd </span><span> # directory, with hostname for the file name and </span><span> # definitions such as: </span><span> # iroute 172.20.0.0 255.255.0.0 </span><span> &quot;--client-config-dir /config/auth/openvpn/ccd&quot;, </span><span> &quot;--keepalive 10 60&quot;, </span><span> &quot;--user nobody&quot;, </span><span> &quot;--group nogroup&quot;, </span><span> &quot;--proto udp&quot;, </span><span> &quot;--port 1195&quot; </span><span> ], </span><span> &quot;server&quot;: { </span><span> &quot;push-route&quot;: [ </span><span> &quot;172.24.0.0/17&quot; </span><span> ], </span><span> &quot;subnet&quot;: &quot;172.24.251.0/24&quot; </span><span> }, </span><span> &quot;tls&quot;: { </span><span> &quot;ca-cert-file&quot;: &quot;/config/auth/openvpn/vps/vps-ca.crt&quot;, </span><span> &quot;cert-file&quot;: &quot;/config/auth/openvpn/vps/vps-server.crt&quot;, </span><span> &quot;dh-file&quot;: &quot;/config/auth/openvpn/dh2048.pem&quot;, </span><span> &quot;key-file&quot;: &quot;/config/auth/openvpn/vps/vps-server.key&quot; </span><span> } </span><span> }, </span></code></pre> <h2 id="netflow">Netflow</h2> <p>Net flows allow a set of connection tracking data to be sent to a remote host for aggregation and analysis. Sadly this process was mostly undocumented, bar some useful forum commentors. Here is the process that I came up with. This is how you configure it live:</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>set system flow-accounting interface eth3.11 </span><span>set system flow-accounting netflow server 172.24.10.22 port 6500 </span><span>set system flow-accounting netflow version 5 </span><span>set system flow-accounting netflow sampling-rate 1 </span><span>set system flow-accounting netflow timeout max-active-life 1 </span><span>commit </span></code></pre> <p>To make this persistent:</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>&quot;system&quot;: { </span><span> &quot;flow-accounting&quot;: { </span><span> &quot;interface&quot;: [ </span><span> &quot;eth3.11&quot;, </span><span> &quot;eth3.12&quot; </span><span> ], </span><span> &quot;netflow&quot;: { </span><span> &quot;sampling-rate&quot;: &quot;1&quot;, </span><span> &quot;version&quot;: &quot;5&quot;, </span><span> &quot;server&quot;: { </span><span> &quot;172.24.10.22&quot;: { </span><span> &quot;port&quot;: &quot;6500&quot; </span><span> } </span><span> }, </span><span> &quot;timeout&quot;: { </span><span> &quot;max-active-life&quot;: &quot;1&quot; </span><span> } </span><span> } </span><span> } </span><span> }, </span></code></pre> <p>To show the current state of your flows:</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>show flow-accounting </span></code></pre> Nextcloud and badrequest filesize incorrect Mon, 31 Dec 2018 00:00:00 +0000 Unknown https://fy.blackhats.net.au/blog/2018-12-31-nextcloud-and-badrequest-filesize-incorrect/ https://fy.blackhats.net.au/blog/2018-12-31-nextcloud-and-badrequest-filesize-incorrect/ <h1 id="nextcloud-and-badrequest-filesize-incorrect">Nextcloud and badrequest filesize incorrect</h1> <p>My friend came to my house and was trying to share some large files with my nextcloud instance. Part way through the upload an error occurred.</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>&quot;Exception&quot;:&quot;Sabre\\DAV\\Exception\\BadRequest&quot;,&quot;Message&quot;:&quot;expected filesize 1768906752 got 1768554496&quot; </span></code></pre> <p>It turns out this error can be caused by many sources. It could be timeouts, bad requests, network packet loss, incorrect nextcloud configuration or more.</p> <p>We tried uploading larger files (by a factor of 10 times) and they worked. This eliminated timeouts as a cause, and probably network loss. Being on ethernet direct to the server generally also helps to eliminate packet loss as a cause compared to say internet.</p> <p>We also knew that the server must not have been misconfigured because a larger file did upload, so no file or resource limits were being hit.</p> <p>This also indicated that the client was likely doing the right thing because larger and smaller files would upload correctly. The symptom now only affected a single file.</p> <p>At this point I realised, what if the client and server were both victims to a lower level issue? I asked my friend to ls the file and read me the number of bytes long. It was 1768906752, as expected in nextcloud.</p> <p>Then I asked him to cat that file into a new file, and to tell me the length of the new file. Cat encountered an error, but ls on the new file indeed showed a size of 1768554496. That means filesystem corruption! What could have lead to this?</p> <p>HFS+</p> <p>Apple's legacy filesystem (and the reason I stopped using macs) is well known for silently eating files and corrupting content. Here we had yet another case of that damage occuring, and triggering errors elsewhere.</p> <p>Bisecting these issues and eliminating possibilities through a scientific method is always the best way to resolve the cause, and it may come from surprising places!</p> Identity ideas ... Fri, 21 Dec 2018 00:00:00 +0000 Unknown https://fy.blackhats.net.au/blog/2018-12-21-identity-ideas/ https://fy.blackhats.net.au/blog/2018-12-21-identity-ideas/ <h1 id="identity-ideas">Identity ideas ...</h1> <p>I've been meaning to write this post for a long time. Taking half a year away from the 389-ds team, and exploring a lot of ideas from other projects has led me to come up with some really interesting ideas about what we do well, and what we don't. I feel like this blog could be divisive, as I really think that for our services to stay relevant we need to make changes that really change our own identity - so that we can better represent yours.</p> <p>So strap in, this is going to be long ...</p> <h2 id="what-s-currently-on-the-market">What's currently on the market</h2> <p>Right now the market for identity has two extremes. At one end we have the legacy &quot;create your own&quot; systems, that are build on technologies like LDAP and Kerberos. I'm thinking about things like 389 Directory Server, OpenLDAP, Active Directory, FreeIPA and more. These all happen to be constrained heavily by complexity, fragility, and administrative workload. You need to spend months to learn these and even still, you will make mistakes and there will be problems.</p> <p>At the other end we have hosted &quot;Identity as a Service&quot; options like Azure AD and Auth0. These have very intelligently, unbound themself from legacy, and tend to offer HTTP apis, 2fa and other features that &quot;just work&quot;. But they are all in the cloud, and outside your control.</p> <p>But there is nothing in the middle. There is no option that &quot;just works&quot;, supports modern standards, and is unhindered by legacy that you can self deploy with minimal administrative fuss - or years of experience.</p> <h2 id="what-do-i-like-from-389">What do I like from 389?</h2> <ul> <li>Replication</li> </ul> <p>The replication system is extremely robust, and has passed many complex tests for cases of eventual consistency correctness. It's very rare to hear of any kind of data corruption or loss within our replication system, and that's testament to the great work of people who spent years looking at the topic.</p> <ul> <li>Performance</li> </ul> <p>We aren't as fast as OpenLDAP is 1 vs 1 server, but our replication scalability is much higher, where in any size of MMR or read-only replica topology, we have higher horizontal scaling, nearly linear based on server additions. If you want to run a cloud scale replicated database, we scale to it (and people already do this!).</p> <ul> <li>Stability</li> </ul> <p>Our server stability is well known with administrators, and honestly is a huge selling point. We see servers that only go down when administrators are performing upgrades. Our work with sanitising tools and the careful eyes of the team has ensured our code base is reliable and solid. Having extensive tests and amazing dedicated quality engineers also goes a long way.</p> <ul> <li>Feature rich</li> </ul> <p>There are a lot of features I really like, and are really useful as an admin deploying this service. Things like memberof (which is actually a group resolution cache when you think about it ...), automember, online backup, unique attribute enforcement, dereferencing, and more.</p> <ul> <li>The team</li> </ul> <p>We have a wonderful team of really smart people, all of whom are caring and want to advance the state of identity management. Not only do they want to keep up with technical changes and excellence, they are listening to and want to improve our social awareness of identity management.</p> <h2 id="pain-points">Pain Points</h2> <ul> <li>C</li> </ul> <p>Because DS is written in C, it's risky and difficult to make changes. People constantly make mistakes that introduce unsafety (even myself), and worse. No amount of tooling or intelligence can take away the fact that C is just hard to use, and people need to be perfect (people are not perfect!) and today we have better tools. We can not spend our time chasing our tails on pointless issues that C creates, when we should be doing better things.</p> <ul> <li>Everything about dynamic admin, config, and plugins is hard and can't scale</li> </ul> <p>Because we need to maintain consistency through operations from start to end but we also allow changing config, plugins, and more during the servers operation the current locking design just doesn't scale. It's also not 100% safe either as the values are changed by atomics, not managed by transactions. We could use copy-on-write for this, but why? Config should be managed by tools like ansible, but today our dynamic config and plugins is both a performance over head and an admin overhead because we exclude best practice tools and have to spend a large amount of time to maintain consistent data when we shouldn't need to. Less features is less support overhead on us, and simpler to test and assert quality and correct behaviour.</p> <ul> <li>Plugins to address shortfalls, but a bit odd.</li> </ul> <p>We have all these features to address issues, but they all do it ... kind of the odd way. Managed Entries creates user private groups on object creation. But the problem is &quot;unix requires a private group&quot; and &quot;ldap schema doesn't allow a user to be a group and user at the same time&quot;. So the answer is actually to create a new objectClass that let's a user ALSO be it's own UPG, not &quot;create an object that links to the user&quot;. (Or have a client generate the group from user attributes but we shouldn't shift responsibility to the client.)</p> <p>Distributed Numeric Assignment is based on the AD rid model, but it's all about &quot;how can we assign a value to a user that's unique?&quot;. We already have a way to do this, in the UUID, so why not derive the UID/GID from the UUID. This means there is no complex inter-server communication, pooling, just simple isolated functionality.</p> <p>We have lots of features that just are a bit complex, and could have been made simpler, that now we have to support, and can't change to make them better. If we rolled a new &quot;fixed&quot; version, we would then have to support both because projects like FreeIPA aren't going to just change over.</p> <ul> <li>client tools are controlled by others and complex (sssd, openldap)</li> </ul> <p>Every tool for dealing with ldap is really confusing and arcane. They all have wild (unhelpful) defaults, and generally this scares people off. I took months of work to get a working ldap server in the past. Why? It's 2018, things need to &quot;just work&quot;. Our tools should &quot;just work&quot;. Why should I need to hand edit pam? Why do I need to set weird options in SSSD.conf? All of this makes the whole experience poor.</p> <p>We are making client tools that can help (to an extent), but they are really limited to system administration and they aren't &quot;generic&quot; tools for every possible configuration that exists. So at some point people will still find a limit where they have to touch ldap commands. A common request is a simple to use web portal for password resets, which today only really exists in FreeIPA, and that limits it's application already.</p> <ul> <li>hard to change legacy</li> </ul> <p>It's really hard to make code changes because our surface area is so broad and the many use cases means that we risk breakage every time we do. I have even broken customer deployments like this. It's almost impossible to get away from, and that holds us back because it means we are scared to make changes because we have to support the 1 million existing work flows. To add another is more support risk.</p> <p>Many deployments use legacy schema elements that holds us back, ranging from the inet types, schema that enforces a first/last name, schema that won't express users + groups in a simple away. It's hard to ask people to just up and migrate their data, and even if we wanted too, ldap allows too much freedom so we are more likely to break data, than migrate it correctly if we tried.</p> <p>This holds us back from technical changes, and social representation changes. People are more likely to engage with a large migrational change, than an incremental change that disturbs their current workflow (IE moving from on prem to cloud, rather than invest in smaller iterative changes to make their local solutions better).</p> <ul> <li>ACI's are really complex</li> </ul> <p>389's access controls are good because they are in the tree and replicated, but bad because the syntax is awful, complex, and has lots of traps and complexity. Even I need to look up how to write them when I have to. This is not good for a project that has such deep security concerns, where your ACI's can look correct but actually expose all your data to risks.</p> <ul> <li>LDAP as a protocol is like an 90's drug experience</li> </ul> <p>LDAP may be the lingua franca of authentication, but it's complex, hard to use and hard to write implementations for. That's why in opensource we have a monoculture of using the openldap client libraries because <em>no one can work out how to write a standalone library</em>. Layer on top the complexity of the object and naming model, and we have a situation where no one wants to interact with LDAP and rather keeps it at arm length.</p> <p>It's going to be extremely hard to move forward here, because the community is so fragmented and small, and the working groups dispersed that the idea of LDAPv4 is a dream that no one should pursue, even though it's desperately needed.</p> <ul> <li>TLS</li> </ul> <p>TLS is great. NSS databases and tools are not.</p> <ul> <li>GSSAPI + SSO</li> </ul> <p>GSSAPI and Kerberos are a piece of legacy that we just can't escape from. They are a curse almost, and one we need to break away from as it's completely unusable (even if it what it promises is amazing). We need to do better.</p> <p>That and SSO allows loads of attacks to proceed, where we actually want isolated token auth with limited access scopes ...</p> <h2 id="what-could-we-offer">What could we offer</h2> <ul> <li>Web application as a first class consumer.</li> </ul> <p>People want web portals for their clients, and they want to be able to use web applications as the consumer of authentication. The HTTP protocols must be the first class integration point for anything in identity management today. This means using things like OAUTH/OIDC.</p> <ul> <li>Systems security as a first class consumer.</li> </ul> <p>Administrators still need to SSH to machines, and people still need their systems to have identities running on them. Having pam/nsswitch modules is a very major requirement, where those modules have to be fast, simple, and work correctly. Users should &quot;imply&quot; a private group, and UID/GID should by dynamic from UUID (or admins can override it).</p> <ul> <li>2FA/u2f/TOTP.</li> </ul> <p>Multi-factor auth is here (not coming, here), and we are behind the game. We already have Apple and MS pushing for webauthn in their devices. We need to be there for these standards to work, and to support the next authentication tool after that.</p> <ul> <li>Good RADIUS integration.</li> </ul> <p>RADIUS is not going away, and is important in education providers and business networks, so RADIUS must &quot;just work&quot;. Importantly, this means mschapv2 which is the universal default for all clients to operate with, which means nthash.</p> <p>However, we can make the nthash unlinked from your normal password, so you can then have wifi password and a seperate loging password. We could even generate an NTHash containing the TOTP token for more high security environments.</p> <ul> <li>better data structure (flat, defined by object types).</li> </ul> <p>The tree structure of LDAP is confusing, but a flatter structure is easier to manage and understand. We can use ideas from kubernetes like tags/labels which can be used to provide certain controls and filtering capabilities for searches and access profiles to apply to.</p> <ul> <li>structured logging, with in built performance profiling.</li> </ul> <p>Being able to diagnose why an operation is slow is critical and having structured logs with profiling information is key to allowing admins and developers to resolve performance issues at scale. It's also critical to have auditing of every single change made in the system, including internal changes that occur during operations.</p> <ul> <li>access profiles with auditing capability.</li> </ul> <p>Access profiles that express what you can access, and how. Easier to audit, generate, and should be tightly linked to group membership for real RBAC style capabilities.</p> <ul> <li>transactions by allowing batch operations.</li> </ul> <p>LDAP wants to provide a transaction system over a set of operations, but that may cause performance issues on write paths. Instead, why not allow submission of batches of changes that all must occur &quot;at the same time&quot; or &quot;none&quot;. This is faster network wise, protocol wise, and simpler for a server to implement.</p> <h2 id="what-s-next-then">What's next then ...</h2> <p>Instead of fixing what we have, why not take the best of what we have, and offer something new in parallel? Start a new front end that speaks in an accessible way, that has modern structures, and has learnt from the lessons of the past? We can build it to standalone, or proxy from the robust core of 389 Directory Server allowing migration paths, but eschew the pain of trying to bring people to the modern world. We can offer something unique, an open source identity system that's easy to use, fast, secure, that you can run on your terms, or in the cloud.</p> <p>This parallel project seems like a good idea ... I wonder what to name it ...</p> Work around docker exec bug Sun, 09 Dec 2018 00:00:00 +0000 Unknown https://fy.blackhats.net.au/blog/2018-12-09-work-around-docker-exec-bug/ https://fy.blackhats.net.au/blog/2018-12-09-work-around-docker-exec-bug/ <h1 id="work-around-docker-exec-bug">Work around docker exec bug</h1> <p>There is currently a docker exec bug in Centos/RHEL 7 that causes errors such as:</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span># docker exec -i -t instance /bin/sh </span><span>rpc error: code = 2 desc = oci runtime error: exec failed: container_linux.go:247: starting container process caused &quot;process_linux.go:110: decoding init error from pipe caused \&quot;read parent: connection reset by peer\&quot;&quot; </span></code></pre> <p>As a work around you can use nsenter instead:</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>PID=docker inspect --format {{.State.Pid}} &lt;name of container&gt; </span><span>nsenter --target $PID --mount --uts --ipc --net --pid /bin/sh </span></code></pre> <p>For more information, you can see the <a href="https://bugzilla.redhat.com/show_bug.cgi?id=1655214">bugreport here.</a></p> High Available RADVD on Linux Thu, 01 Nov 2018 00:00:00 +0000 Unknown https://fy.blackhats.net.au/blog/2018-11-01-high-available-radvd-on-linux/ https://fy.blackhats.net.au/blog/2018-11-01-high-available-radvd-on-linux/ <h1 id="high-available-radvd-on-linux">High Available RADVD on Linux</h1> <p>Recently I was experimenting again with high availability router configurations so that in the cause of an outage or a failover the other router will take over and traffic is still served.</p> <p>This is usually done through protocols like VRRP to allow virtual ips to exist that can be failed between. However with ipv6 one needs to still allow clients to find the router, and in the cause of a failure, the router advertisments still must continue for client renewals.</p> <p>To achieve this we need two parts. A shared Link Local address, and a special RADVD configuration.</p> <p>Because of howe ipv6 routers work, all traffic (even global) is still sent to your link local router. We can use an address like:</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>fe80::1:1 </span></code></pre> <p>This doesn't clash with any reserved or special ipv6 addresses, and it's easy to remember. Because of how link local works, we can put this on many interfaces of the router (many vlans) with no conflict.</p> <p>So now to the two components.</p> <h2 id="keepalived">Keepalived</h2> <p>Keepalived is a VRRP implementation for linux. It has extensive documentation and sometimes uses some implementation specific language, but it works well for what it does.</p> <p>Our configuration looks like:</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span># /etc/keepalived/keepalived.conf </span><span>global_defs { </span><span> vrrp_version 3 </span><span>} </span><span> </span><span>vrrp_sync_group G1 { </span><span> group { </span><span> ipv6_ens256 </span><span> } </span><span>} </span><span> </span><span>vrrp_instance ipv6_ens256 { </span><span> interface ens256 </span><span> virtual_router_id 62 </span><span> priority 50 </span><span> advert_int 1.0 </span><span> virtual_ipaddress { </span><span> fe80::1:1 </span><span> 2001:db8::1 </span><span> } </span><span> nopreempt </span><span> garp_master_delay 1 </span><span>} </span></code></pre> <p>Note that we provide both a global address and an LL address for the failover. This is important for services and DNS for the router to have the global, but you could omit this. The LL address however is critical to this configuration and must be present.</p> <p>Now you can start up vrrp, and you should see one of your two linux machines pick up the address.</p> <h2 id="radvd">RADVD</h2> <p>For RADVD to work, a feature of the 2.x series is required. Packaging this for el7 is out of scope of this post, but fedora ships the version required.</p> <p>The feature is that RADVD can be configured to specify which address it advertises for the router, rather than assuming the interface LL autoconf address is the address to advertise. The configuration appears as:</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span># /etc/radvd.conf </span><span>interface ens256 </span><span>{ </span><span> AdvSendAdvert on; </span><span> MinRtrAdvInterval 30; </span><span> MaxRtrAdvInterval 100; </span><span> AdvRASrcAddress { </span><span> fe80::1:1; </span><span> }; </span><span> prefix 2001:db8::/64 </span><span> { </span><span> AdvOnLink on; </span><span> AdvAutonomous on; </span><span> AdvRouterAddr off; </span><span> }; </span><span>}; </span></code></pre> <p>Note the AdvRASrcAddress parameter? This defines a priority list of address to advertise that could be available on the interface.</p> <p>Now start up radvd on your two routers, and try failing over between them while you ping from your client. Remember to ping LL from a client you need something like:</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>ping6 fe80::1:1%en1 </span></code></pre> <p>Where the outgoing interface of your client traffic is denoted after the '%'.</p> <p>Happy failover routing!</p> Rust RwLock and Mutex Performance Oddities Fri, 19 Oct 2018 00:00:00 +0000 Unknown https://fy.blackhats.net.au/blog/2018-10-19-rust-rwlock-and-mutex-performance-oddities/ https://fy.blackhats.net.au/blog/2018-10-19-rust-rwlock-and-mutex-performance-oddities/ <h1 id="rust-rwlock-and-mutex-performance-oddities">Rust RwLock and Mutex Performance Oddities</h1> <p>Recently I have been working on Rust datastructures once again. In the process I wanted to test how my work performed compared to a standard library RwLock and Mutex. On my home laptop the RwLock was 5 times faster, the Mutex 2 times faster than my work.</p> <p>So checking out my code on my workplace workstation and running my bench marks I noticed the Mutex was the same - 2 times faster. However, the RwLock was 4000 times slower.</p> <h2 id="what-s-a-rwlock-and-mutex-anyway">What's a RwLock and Mutex anyway?</h2> <p>In a multithreaded application, it's important that data that needs to be shared between threads is consistent when accessed. This consistency is not just logical consistency of the data, but affects hardware consistency of the memory in cache. As a simple example, let's examine an update to a bank account done by two threads:</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>acc = 10 </span><span>deposit = 3 </span><span>withdrawl = 5 </span><span> </span><span>[ Thread A ] [ Thread B ] </span><span>acc = load_balance() acc = load_balance() </span><span>acc = acc + deposit acc = acc - withdrawl </span><span>store_balance(acc) store_balance(acc) </span></code></pre> <p>What will the account balance be at the end? The answer is &quot;it depends&quot;. Because threads are working in parallel these operations could happen:</p> <ul> <li>At the same time</li> <li>Interleaved (various possibilities)</li> <li>Sequentially</li> </ul> <p>This isn't very healthy for our bank account. We could lose our deposit, or have invalid data. Valid outcomes at the end are that acc could be 13, 5, 8. Only one of these is correct.</p> <p>A mutex protects our data in multiple ways. It provides hardware consistency operations so that our cpus cache state is valid. It also allows only a single thread inside of the mutex at a time so we can linearise operations. Mutex comes from the word &quot;Mutual Exclusion&quot; after all.</p> <p>So our example with a mutex now becomes:</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>acc = 10 </span><span>deposit = 3 </span><span>withdrawl = 5 </span><span> </span><span>[ Thread A ] [ Thread B ] </span><span>mutex.lock() mutex.lock() </span><span>acc = load_balance() acc = load_balance() </span><span>acc = acc + deposit acc = acc - withdrawl </span><span>store_balance(acc) store_balance(acc) </span><span>mutex.unlock() mutex.unlock() </span></code></pre> <p>Now only one thread will access our account at a time: The other thread will block until the mutex is released.</p> <p>A RwLock is a special extension to this pattern. Where a mutex guarantees single access to the data in a read and write form, a RwLock (Read Write Lock) allows multiple read-only views OR single read and writer access. Importantly when a writer wants to access the lock, all readers must complete their work and &quot;drain&quot;. Once the write is complete readers can begin again. So you can imagine it as:</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>Time -&gt; </span><span> </span><span>T1: -- read --&gt; x </span><span>T3: -- read --&gt; x x -- read --&gt; </span><span>T3: -- read --&gt; x x -- read --&gt; </span><span>T4: | -- write -- | </span><span>T5: x -- read --&gt; </span></code></pre> <h2 id="test-case-for-the-rwlock">Test Case for the RwLock</h2> <p>My test case is simple. Given a set of 12 threads, we spawn:</p> <ul> <li>8 readers. Take a read lock, read the value, release the read lock. If the value == target then stop the thread.</li> <li>4 writers. Take a write lock, read the value. Add one and write. Continue until value == target then stop.</li> </ul> <p>Other conditions:</p> <ul> <li>The test code is identical between Mutex/RwLock (beside the locking costruct)</li> <li>--release is used for compiler optimisations</li> <li>The test hardware is as close as possible (i7 quad core)</li> <li>The tests are run multiple time to construct averages of the performance</li> </ul> <p>The idea being that X target number of writes must occur, while many readers contend as fast as possible on the read. We are pressuring the system of choice between &quot;many readers getting to read fast&quot; or &quot;writers getting priority to drain/block readers&quot;.</p> <p>On OSX given a target of 500 writes, this was able to complete in 0.01 second for the RwLock. (MBP 2011, 2.8GHz)</p> <p>On Linux given a target of 500 writes, this completed in 42 seconds. This is a 4000 times difference. (i7-7700 CPU @ 3.60GHz)</p> <p>All things considered the Linux machine should have an advantage - it's a desktop processor, of a newer generation, and much faster clock speed. So why is the RwLock performance so much different on Linux?</p> <h2 id="to-the-source-code">To the source code!</h2> <p>Examining the <a href="https://github.com/rust-lang/rust/blob/master/src/libstd/sys/unix/rwlock.rs">Rust source code</a> , many OS primitives come from libc. This is because they require OS support to function. RwLock is an example of this as is mutex and many more. The unix implementation for Rust consumes the pthread_rwlock primitive. This means we need to read man pages to understand the details of each.</p> <p>OSX uses FreeBSD userland components, so we can assume they follow the BSD man pages. In the FreeBSD man page for pthread_rwlock_rdlock we see:</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>IMPLEMENTATION NOTES </span><span> </span><span> To prevent writer starvation, writers are favored over readers. </span></code></pre> <p>Linux however, uses different constructs. Looking at the Linux man page:</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>PTHREAD_RWLOCK_PREFER_READER_NP </span><span> This is the default. A thread may hold multiple read locks; </span><span> that is, read locks are recursive. According to The Single </span><span> Unix Specification, the behavior is unspecified when a reader </span><span> tries to place a lock, and there is no write lock but writers </span><span> are waiting. Giving preference to the reader, as is set by </span><span> PTHREAD_RWLOCK_PREFER_READER_NP, implies that the reader will </span><span> receive the requested lock, even if a writer is waiting. As </span><span> long as there are readers, the writer will be starved. </span></code></pre> <h2 id="reader-vs-writer-preferences">Reader vs Writer Preferences?</h2> <p>Due to the policy of a RwLock having multiple readers OR a single writer, a preference is given to one or the other. The preference basically boils down to the choice of:</p> <ul> <li>Do you respond to write requests and have new readers block?</li> <li>Do you favour readers but let writers block until reads are complete?</li> </ul> <p>The difference is that on a <em>read</em> heavy workload, a write will continue to be delayed so that readers can begin <em>and</em> complete (up until some threshold of time). However, on a writer focused workload, you allow readers to stall so that writes can complete sooner.</p> <p>On Linux, they choose a reader preference. On OSX/BSD they choose a writer preference.</p> <p>Because our test is about how fast can a target of write operations complete, the writer preference of BSD/OSX causes this test to be much faster. Our readers still &quot;read&quot; but are giving way to writers, which completes our test sooner.</p> <p>However, the linux &quot;reader favour&quot; policy means that our readers (designed for creating conteniton) are allowed to skip the queue and block writers. This causes our writers to starve. Because the test is only concerned with writer completion, the result is (correctly) showing our writers are heavily delayed - even though many more readers are completing.</p> <p>If we were to track the number of reads that completed, I am sure we would see a large factor of difference where Linux has allow many more readers to complete than the OSX version.</p> <p>Linux pthread_rwlock does allow you to change this policy (PTHREAD_RWLOCK_PREFER_WRITER_NP) but this isn't exposed via Rust. This means today, you accept (and trust) the OS default. Rust is just unaware at compile time and run time that such a different policy exists.</p> <h2 id="conclusion">Conclusion</h2> <p>Rust like any language consumes operating system primitives. Every OS implements these differently and these differences in OS policy can cause real performance differences in applications between development and production.</p> <p>It's well worth understanding the constructions used in programming languages and how they affect the performance of your application - and the decisions behind those tradeoffs.</p> <p>This isn't meant to say &quot;don't use RwLock in Rust on Linux&quot;. This is meant to say &quot;choose it when it makes sense - on read heavy loads, understanding writers will delay&quot;. For my project (A copy on write cell) I will likely conditionally compile rwlock on osx, but mutex on linux as I require a writer favoured behaviour. There are certainly applications that will benefit from the reader priority in linux (especially if there is low writer volume and low penalty to delayed writes).</p> Photography - Why You Should Use JPG (not RAW) Mon, 06 Aug 2018 00:00:00 +0000 Unknown https://fy.blackhats.net.au/blog/2018-08-06-photography-why-you-should-use-jpg-not-raw/ https://fy.blackhats.net.au/blog/2018-08-06-photography-why-you-should-use-jpg-not-raw/ <h1 id="photography-why-you-should-use-jpg-not-raw">Photography - Why You Should Use JPG (not RAW)</h1> <p>When I started my modern journey into photography, I simply shot in JPG. I was happy with the results, and the images I was able to produce. It was only later that I was introduced to a now good friend and he said: &quot;You should always shoot RAW! You can edit so much more if you do.&quot;. It's not hard to find many 'beginner' videos all touting the value of RAW for post editing, and how it's the step from beginner to serious photographer (and editor).</p> <p>Today, I would like to explore why I have turned off RAW on my camera bodies for good. This is a deeply personal decision, and I hope that my experience helps you to think about your own creative choices. If you want to stay shooting RAW and editing - good on you. If this encourages you to try turning back to JPG - good on you too.</p> <p>There are two primary reasons for why I turned off RAW:</p> <ul> <li>Colour reproduction of in body JPG is better to the eye today.</li> <li>Photography is about composing an image from what you have infront of you.</li> </ul> <h2 id="colour-is-about-experts-and-detail">Colour is about experts (and detail)</h2> <p>I have always been unhappy with the colour output of my editing software when processing RAW images. As someone who is colour blind I did not know if it was just my perception, or if real issues existed. No one else complained so it must just be me right!</p> <p>Eventually I stumbled on an article about how to develop real colour and extract camera film simulations for my editor. I was interested in both the ability to get <em>true</em> reflections of colour in my images, but also to use the film simulations in post (the black and white of my camera body is beautiful and soft, but my editor is harsh).</p> <p>I spent a solid week testing and profiling both of my cameras. I quickly realised a great deal about what was occuring in my editor, but also my camera body.</p> <p>The editor I have, is attempting to generalise over the entire set of sensors that a manufacturer has created. They are also attempting to create a <em>true</em> colour output profile, that is as reflective of reality as possible. So when I was exporting RAWs to JPG, I was seeing the <em>differences</em> between what my camera hardware is, vs the editors profiles. (This was particularly bad on my older body, so I suspect the RAW profiles are designed for the newer sensor).</p> <p>I then created film simulations and quickly noticed the subtle changes. Blacks were blacker, but retained more fine detail with the simulation. Skin tone was softer. Exposure was more even across a variety of image types. How? RAW and my editor is meant to create the best image possible? Why is a film-simulation I have &quot;extracted&quot; creating better images?</p> <p>As any good engineer would do I created sample images. A/B testing. I would provide the RAW processed by my editor, and a RAW processed with my film simulation. I would vary the left/right of the image, exposure, subject, and more. After about 10 tests across 5 people, only on one occasion did someone prefer the RAW from my editor.</p> <p>At this point I realised that my camera manufacturer is hiring experts who build, live and breath colour technology. They have tested and examined everything about the body I have, and likely calibrated it individually in the process to make produce exact reproductions as they see in a lab. They are developing colour profiles that are not just broadly applicable, but also <em>pleasing</em> to look at (even if not accurate reproductions).</p> <p>So how can my film simulations I extracted and built in a week, measure up to the experts? I decided to find out. I shot test images in JPG and RAW and began to provide A/B/C tests to people.</p> <p>If the editor RAW was washed out compared to the RAW with my film simulation, the JPG from the body made them pale in comparison. Every detail was better, across a range of conditions. The features in my camera body are <em>better</em> than my editor. Noise reduction, dynamic range, sharpening, softening, colour saturation. I was holding in my hands a device that has thousands of hours of expert design, that could eclipse anything I built on a weekend for fun to achieve the same.</p> <p>It was then I came to think about and realise ...</p> <h2 id="composition-and-effects-is-about-you">Composition (and effects) is about you</h2> <p>Photography is a complex skill. It's not having a fancy camera and just clicking the shutter, or zooming in. Photography is about taking that camera and putting it in a position to take a well composed image based on many rules (and exceptions) that I am still continually learning.</p> <p>When you stop to look at an image you should always think &quot;how can I compose the best image possible?&quot;.</p> <p>So why shoot in RAW? RAW is all about enabling editing in <em>post</em>. After you have already composed and taken the image. There are valid times and useful functions of editing. For example whitebalance correction and minor cropping in some cases. Both of these are easily conducted with JPG with no loss in quality compared to the RAW. I still commonly do both of these.</p> <p>However RAW allows you to recover mistakes during composition (to a point). For example, the powerful base-curve fusion module allows dynamic range &quot;after the fact&quot;. You may even add high or low pass filters, or mask areas to filter and affect the colour to make things pop, or want that RAW data to make your vibrance control as perfect as possible. You may change the perspective, or even add filters and more. Maybe you want to optimise de-noise to make smooth high ISO images. There are so many options!</p> <p>But all these things are you composing <em>after</em>. Today, many of these functions are in your camera - and better performing. So while I'm composing I can enable dynamic range for the darker elements of the frame. I can compose and add my colour saturation (or remove it). I can sharpen, soften. I can move my own body to change perspective. All at the time I am building the image in my mind, as I compose, I am able to decide on the creative effects I want to place in that image. I'm not longer just composing within a frame, but a canvas of potential effects.</p> <p>To me this was an important distinction. I always found I was editing poorly-composed images in an attempt to &quot;fix&quot; them to something acceptable. Instead I should have been looking at how to compose them from the start to be great, using the tool in my hand - my camera.</p> <p>Really, this is a decision that is yours. Do you spend more time now to make the image you want? Or do you spend it later editing to achieve what you want?</p> <h2 id="conclusion">Conclusion</h2> <p>Photography is a creative process. You will have your own ideas of how that process should look, and how you want to work with it. Great! This was my experience and how I have arrived at a creative process that I am satisfied with. I hope that it provides you an alternate perspective to the generally accepted &quot;RAW is imperative&quot; line that many people advertise.</p> Extracting Formally Verified C with FStar and KreMLin Mon, 30 Apr 2018 00:00:00 +0000 Unknown https://fy.blackhats.net.au/blog/2018-04-30-extracting-formally-verified-c-with-fstar-and-kremlin/ https://fy.blackhats.net.au/blog/2018-04-30-extracting-formally-verified-c-with-fstar-and-kremlin/ <h1 id="extracting-formally-verified-c-with-fstar-and-kremlin">Extracting Formally Verified C with FStar and KreMLin</h1> <p>As software engineering has progressed, the correctness of software has become a major issue. However the tools that exist today to help us create correct programs have been lacking. Human's continue to make mistakes that cause harm to others (even I do!).</p> <p>Instead, tools have now been developed that allow the verification of programs and algorithms as correct. These have not gained widespread adoption due to the complexities of their tool chains or other social and cultural issues.</p> <p>The Project Everest research has aimed to create a formally verified webserver and cryptography library. To achieve this they have developed a language called F* (FStar) and KreMLin as an extraction tool. This allows an FStar verified algorithm to be extracted to a working set of C source code - C source code that can be easily added to existing projects.</p> <h2 id="setting-up-a-fstar-and-kremlin-environment">Setting up a FStar and KreMLin environment</h2> <p>Today there are a number of undocumented gotchas with opam - the OCaml package manager. Most of these are silent errors. I used the following steps to get to a working environment.</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span># It&#39;s important to have bzip2 here else opam silently fails! </span><span>dnf install -y rsync git patch opam bzip2 which gmp gmp-devel m4 \ </span><span> hg unzip pkgconfig redhat-rpm-config </span><span> </span><span>opam init </span><span># You need 4.02.3 else wasm will not compile. </span><span>opam switch create 4.02.3 </span><span>opam switch 4.02.3 </span><span>echo &quot;. /home/Work/.opam/opam-init/init.sh &gt; /dev/null 2&gt; /dev/null || true&quot; &gt;&gt; .bashrc </span><span>opam install batteries fileutils yojson ppx_deriving_yojson zarith fix pprint menhir process stdint ulex wasm </span><span> </span><span>PATH &quot;~/z3/bin:~/FStar/bin:~/kremlin:$PATH&quot; </span><span># You can get the &quot;correct&quot; z3 version from https://github.com/FStarLang/binaries/raw/master/z3-tested/z3-4.5.1.1f29cebd4df6-x64-ubuntu-14.04.zip </span><span>unzip z3-4.5.1.1f29cebd4df6-x64-ubuntu-14.04.zip </span><span>mv z3-4.5.1.1f29cebd4df6-x64-ubuntu-14.04 z3 </span><span> </span><span># You will need a &quot;stable&quot; release of FStar https://github.com/FStarLang/FStar/archive/stable.zip </span><span>unzip stable.zip </span><span>mv FStar-stable Fstar </span><span>cd ~/FStar </span><span>opam config exec -- make -C src/ocaml-output -j </span><span>opam config exec -- make -C ulib/ml </span><span> </span><span># You need a master branch of kremlin https://github.com/FStarLang/kremlin/archive/master.zip </span><span>cd ~ </span><span>unzip master.zip </span><span>mv kremlin-master kremlin </span><span>cd kremlin </span><span>opam config exec -- make </span><span>opam config exec -- make test </span></code></pre> <h2 id="your-first-fstar-extraction">Your first FStar extraction</h2> AD directory admins group setup Thu, 26 Apr 2018 00:00:00 +0000 Unknown https://fy.blackhats.net.au/blog/2018-04-26-ad-directory-admins-group-setup/ https://fy.blackhats.net.au/blog/2018-04-26-ad-directory-admins-group-setup/ <h1 id="ad-directory-admins-group-setup">AD directory admins group setup</h1> <p>Recently I have been reading many of the Microsoft Active Directory best practices for security and hardening. These are great resources, and very well written. The major theme of the articles is &quot;least privilege&quot;, where accounts like Administrators or Domain Admins are over used and lead to further compromise.</p> <p>A suggestion that is put forward by the author is to have a group that has no other permissions but to manage the directory service. This should be used to temporarily make a user an admin, then after a period of time they should be removed from the group.</p> <p>This way you have no Administrators or Domain Admins, but you have an AD only group that can temporarily grant these permissions when required.</p> <p>I want to explore how to create this and configure the correct access controls to enable this scheme.</p> <h2 id="create-our-group">Create our group</h2> <p>First, lets create a &quot;Directory Admins&quot; group which will contain our members that have the rights to modify or grant other privileges.</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span># /usr/local/samba/bin/samba-tool group add &#39;Directory Admins&#39; </span><span>Added group Directory Admins </span></code></pre> <p>It's a really good idea to add this to the &quot;Denied RODC Password Replication Group&quot; to limit the risk of these accounts being compromised during an attack. Additionally, you probably want to make your &quot;admin storage&quot; group also a member of this, but I'll leave that to you.</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span># /usr/local/samba/bin/samba-tool group addmembers &quot;Denied RODC Password Replication Group&quot; &quot;Directory Admins&quot; </span></code></pre> <p>Now that we have this, lets add a member to it. I strongly advise you create special accounts just for the purpose of directory administration - don't use your daily account for this!</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span># /usr/local/samba/bin/samba-tool user create da_william </span><span>User &#39;da_william&#39; created successfully </span><span># /usr/local/samba/bin/samba-tool group addmembers &#39;Directory Admins&#39; da_william </span><span>Added members to group Directory Admins </span></code></pre> <h2 id="configure-the-permissions">Configure the permissions</h2> <p>Now we need to configure the correct dsacls to allow Directory Admins full control over directory objects. It could be possible to constrain this to <em>only</em> modification of the cn=builtin and cn=users container however, as directory admins might not need so much control for things like dns modification.</p> <p>If you want to constrain these permissions, only apply the following to cn=builtins instead - or even just the target groups like Domain Admins.</p> <p>First we need the objectSID of our Directory Admins group so we can build the ACE.</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span># /usr/local/samba/bin/samba-tool group show &#39;directory admins&#39; --attributes=cn,objectsid </span><span>dn: CN=Directory Admins,CN=Users,DC=adt,DC=blackhats,DC=net,DC=au </span><span>cn: Directory Admins </span><span>objectSid: S-1-5-21-2488910578-3334016764-1009705076-1104 </span></code></pre> <p>Now with this we can construct the ACE.</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>(A;CI;RPWPLCLORC;;;S-1-5-21-2488910578-3334016764-1009705076-1104) </span></code></pre> <p>This permission grants:</p> <ul> <li>RP: read property</li> <li>WP: write property</li> <li>LC: list child objects</li> <li>LO: list objects</li> <li>RC: read control</li> </ul> <p>It could be possible to expand these rights: it depends if you want directory admins to be able to do &quot;day to day&quot; ad control jobs, or if you just use them for granting of privileges. That's up to you. An expanded ACE might be:</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span># Same as Enterprise Admins </span><span>(A;CI;RPWPCRCCDCLCLORCWOWDSW;;;S-1-5-21-2488910578-3334016764-1009705076-1104) </span></code></pre> <p>Now lets actually apply this and do a test:</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span># /usr/local/samba/bin/samba-tool dsacl set --sddl=&#39;(A;CI;RPWPLCLORC;;;S-1-5-21-2488910578-3334016764-1009705076-1104)&#39; --objectdn=&#39;dc=adt,dc=blackhats,dc=net,dc=au&#39; </span><span># /usr/local/samba/bin/samba-tool group addmembers &#39;directory admins&#39; administrator -U &#39;da_william%...&#39; </span><span>Added members to group directory admins </span><span># /usr/local/samba/bin/samba-tool group listmembers &#39;directory admins&#39; -U &#39;da_william%...&#39; </span><span>da_william </span><span>Administrator </span></code></pre> <p>After we have completed our tasks, we remove da_william from the directory admins group as we no longer required the privileges. You can self-remove, or have the Administrator account do the removal.</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span># /usr/local/samba/bin/samba-tool group removemembers &#39;directory admins&#39; da_william -U &#39;da_william%...&#39; </span><span>Removed members from group directory admins </span><span> </span><span># /usr/local/samba/bin/samba-tool group removemembers &#39;directory admins&#39; da_william -U &#39;Administrator&#39; </span><span>Removed members from group directory admins </span></code></pre> <p>Finally check that da_william is no longer in the group.</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span># /usr/local/samba/bin/samba-tool group listmembers &#39;directory admins&#39; -U &#39;da_william%...&#39; </span><span>Administrator </span></code></pre> <h2 id="conclusion">Conclusion</h2> <p>With these steps we have created a secure account that has limited admin rights, able to temporarily promote users with privileges for administrative work - and able to remove it once the work is complete.</p> Understanding AD Access Control Entries Fri, 20 Apr 2018 00:00:00 +0000 Unknown https://fy.blackhats.net.au/blog/2018-04-20-understanding-ad-access-control-entries/ https://fy.blackhats.net.au/blog/2018-04-20-understanding-ad-access-control-entries/ <h1 id="understanding-ad-access-control-entries">Understanding AD Access Control Entries</h1> <p>A few days ago I set out to work on making samba 4 my default LDAP server. In the process I was forced to learn about Active Directory Access controls. I found that while there was significant documentation around the syntax of these structures, very little existed explaining how to use them effectively.</p> <h2 id="what-s-in-an-ace">What's in an ACE?</h2> <p>If you look at the the ACL of an entry in AD you'll see something like:</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>O:DAG:DAD:AI </span><span>(A;CI;RPLCLORC;;;AN) </span><span>(A;;RPWPCRCCDCLCLORCWOWDSDDTSW;;;SY) </span><span>(A;;RPWPCRCCDCLCLORCWOWDSW;;;DA) </span><span>(OA;;CCDC;bf967aba-0de6-11d0-a285-00aa003049e2;;AO) </span><span>(OA;;CCDC;bf967a9c-0de6-11d0-a285-00aa003049e2;;AO) </span><span>(OA;;CCDC;bf967aa8-0de6-11d0-a285-00aa003049e2;;PO) </span><span>(A;;RPLCLORC;;;AU) </span><span>(OA;;CCDC;4828cc14-1437-45bc-9b07-ad6f015e5f28;;AO) </span><span>(OA;CIIOID;RP;4c164200-20c0-11d0-a768-00aa006e0529;4828cc14-1437-45bc-9b07-ad6f015e5f28;RU) </span><span>(OA;CIIOID;RP;4c164200-20c0-11d0-a768-00aa006e0529;bf967aba-0de6-11d0-a285-00aa003049e2;RU) </span><span>(OA;CIIOID;RP;5f202010-79a5-11d0-9020-00c04fc2d4cf;4828cc14-1437-45bc-9b07-ad6f015e5f28;RU) </span><span>(OA;CIIOID;RP;5f202010-79a5-11d0-9020-00c04fc2d4cf;bf967aba-0de6-11d0-a285-00aa003049e2;RU) </span><span>(OA;CIIOID;RP;bc0ac240-79a9-11d0-9020-00c04fc2d4cf;4828cc14-1437-45bc-9b07-ad6f015e5f28;RU) </span><span>(OA;CIIOID;RP;bc0ac240-79a9-11d0-9020-00c04fc2d4cf;bf967aba-0de6-11d0-a285-00aa003049e2;RU) </span><span>(OA;CIIOID;RP;59ba2f42-79a2-11d0-9020-00c04fc2d3cf;4828cc14-1437-45bc-9b07-ad6f015e5f28;RU) </span><span>(OA;CIIOID;RP;59ba2f42-79a2-11d0-9020-00c04fc2d3cf;bf967aba-0de6-11d0-a285-00aa003049e2;RU) </span><span>(OA;CIIOID;RP;037088f8-0ae1-11d2-b422-00a0c968f939;4828cc14-1437-45bc-9b07-ad6f015e5f28;RU) </span><span>(OA;CIIOID;RP;037088f8-0ae1-11d2-b422-00a0c968f939;bf967aba-0de6-11d0-a285-00aa003049e2;RU) </span><span>(OA;CIIOID;RP;b7c69e6d-2cc7-11d2-854e-00a0c983f608;bf967a86-0de6-11d0-a285-00aa003049e2;ED) </span><span>(OA;CIIOID;RP;b7c69e6d-2cc7-11d2-854e-00a0c983f608;bf967a9c-0de6-11d0-a285-00aa003049e2;ED) </span><span>(OA;CIIOID;RP;b7c69e6d-2cc7-11d2-854e-00a0c983f608;bf967aba-0de6-11d0-a285-00aa003049e2;ED) </span><span>(OA;CIIOID;RPLCLORC;;4828cc14-1437-45bc-9b07-ad6f015e5f28;RU) </span><span>(OA;CIIOID;RPLCLORC;;bf967a9c-0de6-11d0-a285-00aa003049e2;RU) </span><span>(OA;CIIOID;RPLCLORC;;bf967aba-0de6-11d0-a285-00aa003049e2;RU) </span><span>(OA;CIID;RPWPCR;91e647de-d96f-4b70-9557-d63ff4f3ccd8;;PS) </span><span>(A;CIID;RPWPCRCCDCLCLORCWOWDSDDTSW;;;EA) </span><span>(A;CIID;LC;;;RU) </span><span>(A;CIID;RPWPCRCCLCLORCWOWDSDSW;;;BA) </span><span>S:AI </span><span>(OU;CIIOIDSA;WP;f30e3bbe-9ff0-11d1-b603-0000f80367c1;bf967aa5-0de6-11d0-a285-00aa003049e2;WD) </span><span>(OU;CIIOIDSA;WP;f30e3bbf-9ff0-11d1-b603-0000f80367c1;bf967aa5-0de6-11d0-a285-00aa003049e2;WD) </span></code></pre> <p>This seems very confusing and complex (and someone should write a tool to explain these ... maybe me). But once you can see the structure it starts to make sense.</p> <p>Most of the access controls you are viewing here are DACLs or Discrestionary Access Control Lists. These make up the majority of the output after 'O:DAG:DAD:AI'. TODO: What does 'O:DAG:DAD:AI' mean completely?</p> <p>After that there are many ACEs defined in SDDL or ???. The structure is as follows:</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>(type;flags;rights;object_guid;inherit_object_guid;sid(;attribute)) </span></code></pre> <p>Each of these fields can take varies types. These interact to form the access control rules that allow or deny access. Thankfully, you don't need to adjust many fields to make useful ACE entries.</p> <p>MS maintains a document of these <a href="https://msdn.microsoft.com/en-us/library/aa374928(d=printer,v=vs.85).aspx">field values here.</a></p> <p>They also maintain a list of wellknown <a href="https://msdn.microsoft.com/en-us/library/aa379602(d=printer,v=vs.85).aspx">SID values here</a></p> <p>I want to cover some common values you may see though:</p> <h2 id="type">type</h2> <p>Most of the types you'll see are &quot;A&quot; and &quot;OA&quot;. These mean the ACE allows an access by the SID.</p> <h2 id="flags">flags</h2> <p>These change the behaviour of the ACE. Common values you may want to set are CI and OI. These determine that the ACE should be inherited to child objects. As far as the MS docs say, these behave the same way.</p> <p>If you see ID in this field it means the ACE has been inherited from a parent object. In this case the inherit_object_guid field will be set to the guid of the parent that set the ACE. This is great, as it allows you to backtrace the origin of access controls!</p> <h2 id="rights">rights</h2> <p>This is the important part of the ACE - it determines what access the SID has over this object. The MS docs are very comprehensive of what this does, but common values are:</p> <ul> <li>RP: read property</li> <li>WP: write property</li> <li>CR: control rights</li> <li>CC: child create (create new objects)</li> <li>DC: delete child</li> <li>LC: list child objects</li> <li>LO: list objects</li> <li>RC: read control</li> <li>WO: write owner (change the owner of an object)</li> <li>WD: write dac (allow writing ACE)</li> <li>SW: self write</li> <li>SD: standard delete</li> <li>DT: delete tree</li> </ul> <p>I'm not 100% sure of all the subtle behaviours of these, because they are <em>not</em> documented that well. If someone can help explain these to me, it would be great.</p> <h2 id="sid">sid</h2> <p>We will skip some fields and go straight to SID. This is the SID of the object that is allowed the rights from the rights field. This field can take a GUID of the object, or it can take a &quot;well known&quot; value of the SID. For example 'AN' means &quot;anonymous users&quot;, or 'AU' meaning authenticated users.</p> <h2 id="conclusion">conclusion</h2> <p>I won't claim to be an AD ACE expert, but I did find the docs hard to interpret at first. Having a breakdown and explanation of the behaviour of the fields can help others, and I really want to hear from people who know more about this topic on me so that I can expand this resource to help others really understand how AD ACE's work.</p> Making Samba 4 the default LDAP server Wed, 18 Apr 2018 00:00:00 +0000 Unknown https://fy.blackhats.net.au/blog/2018-04-18-making-samba-4-the-default-ldap-server/ https://fy.blackhats.net.au/blog/2018-04-18-making-samba-4-the-default-ldap-server/ <h1 id="making-samba-4-the-default-ldap-server">Making Samba 4 the default LDAP server</h1> <p>Earlier this year Andrew Bartlett set me the challenge: how could we make Samba 4 the default LDAP server in use for Linux and UNIX systems? I've finally decided to tackle this, and write up some simple changes we can make, and decide on some long term goals to make this a reality.</p> <h2 id="what-makes-a-unix-directory-anyway">What makes a unix directory anyway?</h2> <p>Great question - this is such a broad topic, even I don't know if I can single out what it means. For the purposes of this exercise I'll treat it as &quot;what would we need from my previous workplace&quot;. My previous workplace had a dedicated set of 389 Directory Server machines that served lookups mainly for email routing, application authentication and more. The didn't really process a great deal of login traffic as the majority of the workstations were Windows - thus connected to AD.</p> <p>What it did show was that Linux clients and applications:</p> <ul> <li>Want to use anonymous binds and searchs - Applications and clients are NOT domain members - they just want to do searches</li> <li>The content of anonymous lookups should be &quot;public safe&quot; information. (IE nothing private)</li> <li>LDAPS is a must for binds</li> <li>MemberOf and group filtering is very important for access control</li> <li>sshPublicKey and userCertificate;binary is important for 2fa/secure logins</li> </ul> <p>This seems like a pretty simple list - but it's not the model Samba 4 or AD ship with.</p> <p>You'll also want to harden a few default settings. These include:</p> <ul> <li>Disable Guest</li> <li>Disable 10 machine join policy</li> </ul> <p>AD works under the assumption that all clients are authenticated via kerberos, and that kerberos is the primary authentication and trust provider. As a result, AD often ships with:</p> <ul> <li>Disabled anonymous binds - All clients are domain members or service accounts</li> <li>No anonymous content available to search</li> <li>No LDAPS (GSSAPI is used instead)</li> <li>no sshPublicKey or userCertificates (pkinit instead via krb)</li> <li>Access control is much more complex topic than just &quot;matching an ldap filter&quot;.</li> </ul> <p>As a result, it takes a bit of effort to change Samba 4 to work in a way that suits both, securely.</p> <h2 id="isn-t-anonymous-binding-insecure">Isn't anonymous binding insecure?</h2> <p>Let's get this one out the way - no it's not. In every pen test I have seen if you can get access to a domain joined machine, you probably have a good chance of taking over the domain in various ways. Domain joined systems and krb allows lateral movement and other issues that are beyond the scope of this document.</p> <p>The lack of anonymous lookup is more about preventing information disclosure - security via obscurity. But it doesn't take long to realise that this is trivially defeated (get one user account, guest account, domain member and you can search ...).</p> <p>As a result, in some cases it may be better to allow anonymous lookups because then you don't have spurious service accounts, you have a clear understanding of what is and is not accessible as readable data, and you <em>don't</em> need every machine on the network to be domain joined - you prevent a possible foothold of lateral movement.</p> <p>So anonymous binding is just fine, as the unix world has shown for a long time. That's why I have very few concerns about enabling it. Your safety is in the access controls for searches, not in blocking anonymous reads outright.</p> <h2 id="installing-your-dc">Installing your DC</h2> <p>As I run fedora, you will need to build and install samba for source so you can access the heimdal kerberos functions. Fedora's samba 4 ships ADDC support now, but lacks some features like RODC that you may want. In the future I expect this will change though.</p> <p>These documents will help guide you:</p> <p><a href="https://wiki.samba.org/index.php/Package_Dependencies_Required_to_Build_Samba#Fedora_26">requirements</a></p> <p><a href="https://wiki.samba.org/index.php/Build_Samba_from_Source#Introduction">build steps</a></p> <p><a href="https://wiki.samba.org/index.php/Setting_up_Samba_as_an_Active_Directory_Domain_Controller">install a domain</a></p> <p>I strongly advise you use options similar to:</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>/usr/local/samba/bin/samba-tool domain provision --server-role=dc --use-rfc2307 --dns-backend=SAMBA_INTERNAL --realm=SAMDOM.EXAMPLE.COM --domain=SAMDOM --adminpass=Passw0rd </span></code></pre> <h2 id="allow-anonymous-binds-and-searches">Allow anonymous binds and searches</h2> <p>Now that you have a working domain controller, we should test you have working ldap:</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>/usr/local/samba/bin/samba-tool forest directory_service dsheuristics 0000002 -H ldaps://localhost --simple-bind-dn=&#39;administrator@samdom.example.com&#39; </span><span> </span><span>ldapsearch -b DC=samdom,DC=example,DC=com -H ldaps://localhost -x </span></code></pre> <p>You can see the domain object but nothing else. Many other blogs and sites recommend a blanket &quot;anonymous read all&quot; access control, but I think that's too broad. A better approach is to add the anonymous read to only the few containers that require it.</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>/usr/local/samba/bin/samba-tool dsacl set --objectdn=DC=samdom,DC=example,DC=com --sddl=&#39;(A;;RPLCLORC;;;AN)&#39; --simple-bind-dn=&quot;administrator@samdom.example.com&quot; --password=Passw0rd </span><span>/usr/local/samba/bin/samba-tool dsacl set --objectdn=CN=Users,DC=samdom,DC=example,DC=com --sddl=&#39;(A;CI;RPLCLORC;;;AN)&#39; --simple-bind-dn=&quot;administrator@samdom.example.com&quot; --password=Passw0rd </span><span>/usr/local/samba/bin/samba-tool dsacl set --objectdn=CN=Builtin,DC=samdom,DC=example,DC=com --sddl=&#39;(A;CI;RPLCLORC;;;AN)&#39; --simple-bind-dn=&quot;administrator@samdom.example.com&quot; --password=Passw0rd </span></code></pre> <p>In AD groups and users are found in cn=users, and some groups are in cn=builtin. So we allow read to the root domain object, then we set a read on cn=users and cn=builtin that inherits to it's child objects. The attribute policies are derived elsewhere, so we can assume that things like kerberos data and password material are safe with these simple changes.</p> <h2 id="configuring-ldaps">Configuring LDAPS</h2> <p>This is a reasonable simple exercise. Given a ca cert, key and cert we can place these in the correct locations samba expects. By default this is the private directory. In a custom install, that's /usr/local/samba/private/tls/, but for distros I think it's /var/lib/samba/private. Simply replace ca.pem, cert.pem and key.pem with your files and restart.</p> <h2 id="adding-schema">Adding schema</h2> <p>To allow adding schema to samba 4 you need to reconfigure the dsdb config on the schema master. To show the current schema master you can use:</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>/usr/local/samba/bin/samba-tool fsmo show -H ldaps://localhost --simple-bind-dn=&#39;administrator@adt.blackhats.net.au&#39; --password=Password1 </span></code></pre> <p>Look for the value:</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>SchemaMasterRole owner: CN=NTDS Settings,CN=LDAPKDC,CN=Servers,CN=Default-First-Site-Name,CN=Sites,CN=Configuration,DC=adt,DC=blackhats,DC=net,DC=au </span></code></pre> <p>And note the CN=ldapkdc = that's the hostname of the current schema master.</p> <p>On the schema master we need to adjust the smb.conf. The change you need to make is:</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>[global] </span><span> dsdb:schema update allowed = yes </span></code></pre> <p>Now restart the instance and we can update the schema. The following LDIF should work if you replace ${DOMAINDN} with your namingContext. You can apply it with ldapmodify</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>dn: CN=sshPublicKey,CN=Schema,CN=Configuration,DC=adt,DC=blackhats,DC=net,DC=au </span><span>changetype: add </span><span>objectClass: top </span><span>objectClass: attributeSchema </span><span>attributeID: 1.3.6.1.4.1.24552.500.1.1.1.13 </span><span>cn: sshPublicKey </span><span>name: sshPublicKey </span><span>lDAPDisplayName: sshPublicKey </span><span>description: MANDATORY: OpenSSH Public key </span><span>attributeSyntax: 2.5.5.10 </span><span>oMSyntax: 4 </span><span>isSingleValued: FALSE </span><span>searchFlags: 8 </span><span> </span><span>dn: CN=ldapPublicKey,CN=Schema,CN=Configuration,DC=adt,DC=blackhats,DC=net,DC=au </span><span>changetype: add </span><span>objectClass: top </span><span>objectClass: classSchema </span><span>governsID: 1.3.6.1.4.1.24552.500.1.1.2.0 </span><span>cn: ldapPublicKey </span><span>name: ldapPublicKey </span><span>description: MANDATORY: OpenSSH LPK objectclass </span><span>lDAPDisplayName: ldapPublicKey </span><span>subClassOf: top </span><span>objectClassCategory: 3 </span><span>defaultObjectCategory: CN=ldapPublicKey,CN=Schema,CN=Configuration,DC=adt,DC=blackhats,DC=net,DC=au </span><span>mayContain: sshPublicKey </span><span> </span><span>dn: CN=User,CN=Schema,CN=Configuration,DC=adt,DC=blackhats,DC=net,DC=au </span><span>changetype: modify </span><span>replace: auxiliaryClass </span><span>auxiliaryClass: ldapPublicKey </span><span>auxiliaryClass: posixAccount </span><span>auxiliaryClass: shadowAccount </span><span>- </span><span> </span><span>sudo ldapmodify -f sshpubkey.ldif -D &#39;administrator@adt.blackhats.net.au&#39; -w Password1 -H ldaps://localhost </span><span>adding new entry &quot;CN=sshPublicKey,CN=Schema,CN=Configuration,DC=adt,DC=blackhats,DC=net,DC=au&quot; </span><span> </span><span>adding new entry &quot;CN=ldapPublicKey,CN=Schema,CN=Configuration,DC=adt,DC=blackhats,DC=net,DC=au&quot; </span><span> </span><span>modifying entry &quot;CN=User,CN=Schema,CN=Configuration,DC=adt,DC=blackhats,DC=net,DC=au&quot; </span></code></pre> <p>To my surprise, userCertificate already exists! The reason I missed it is a subtle ad schema behaviour I missed. The <em>ldap attribute</em> name is stored in the lDAPDisplayName and may not be the same as the CN of the schema element. As a result, you can find this with:</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>ldapsearch -H ldaps://localhost -b CN=Schema,CN=Configuration,DC=adt,DC=blackhats,DC=net,DC=au -x -D &#39;administrator@adt.blackhats.net.au&#39; -W &#39;(attributeId=2.5.4.36)&#39; </span></code></pre> <p>This doesn't solve my issues: Because I am a long time user of 389-ds, that means I need some ns compat attributes. Here I add the nsUniqueId value so that I can keep some compatability.</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>dn: CN=nsUniqueId,CN=Schema,CN=Configuration,DC=adt,DC=blackhats,DC=net,DC=au </span><span>changetype: add </span><span>objectClass: top </span><span>objectClass: attributeSchema </span><span>attributeID: 2.16.840.1.113730.3.1.542 </span><span>cn: nsUniqueId </span><span>name: nsUniqueId </span><span>lDAPDisplayName: nsUniqueId </span><span>description: MANDATORY: nsUniqueId compatability </span><span>attributeSyntax: 2.5.5.10 </span><span>oMSyntax: 4 </span><span>isSingleValued: TRUE </span><span>searchFlags: 9 </span><span> </span><span>dn: CN=nsOrgPerson,CN=Schema,CN=Configuration,DC=adt,DC=blackhats,DC=net,DC=au </span><span>changetype: add </span><span>objectClass: top </span><span>objectClass: classSchema </span><span>governsID: 2.16.840.1.113730.3.2.334 </span><span>cn: nsOrgPerson </span><span>name: nsOrgPerson </span><span>description: MANDATORY: Netscape DS compat person </span><span>lDAPDisplayName: nsOrgPerson </span><span>subClassOf: top </span><span>objectClassCategory: 3 </span><span>defaultObjectCategory: CN=nsOrgPerson,CN=Schema,CN=Configuration,DC=adt,DC=blackhats,DC=net,DC=au </span><span>mayContain: nsUniqueId </span><span> </span><span>dn: CN=User,CN=Schema,CN=Configuration,DC=adt,DC=blackhats,DC=net,DC=au </span><span>changetype: modify </span><span>replace: auxiliaryClass </span><span>auxiliaryClass: ldapPublicKey </span><span>auxiliaryClass: posixAccount </span><span>auxiliaryClass: shadowAccount </span><span>auxiliaryClass: nsOrgPerson </span><span>- </span></code></pre> <p>Now with this you can extend your users with the required data for SSH, certificates and maybe 389-ds compatability.</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>/usr/local/samba/bin/samba-tool user edit william -H ldaps://localhost --simple-bind-dn=&#39;administrator@adt.blackhats.net.au&#39; </span></code></pre> <h2 id="performance">Performance</h2> <p>Out of the box a number of the unix attributes are not indexed by Active Directory. To fix this you need to update the search flags in the schema.</p> <p>Again, temporarily allow changes:</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>[global] </span><span> dsdb:schema update allowed = yes </span></code></pre> <p>Now we need to add some indexes for common types. Note that in the nsUniqueId schema I already added the search flags. We also want to set that these values should be preserved if they become tombstones so we can recove them.</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>/usr/local/samba/bin/samba-tool schema attribute modify uid --searchflags=9 </span><span>/usr/local/samba/bin/samba-tool schema attribute modify nsUniqueId --searchflags=9 </span><span>/usr/local/samba/bin/samba-tool schema attribute modify uidnumber --searchflags=9 </span><span>/usr/local/samba/bin/samba-tool schema attribute modify gidnumber --searchflags=9 </span><span># Preserve on tombstone but don&#39;t index </span><span>/usr/local/samba/bin/samba-tool schema attribute modify x509-cert --searchflags=8 </span><span>/usr/local/samba/bin/samba-tool schema attribute modify sshPublicKey --searchflags=8 </span><span>/usr/local/samba/bin/samba-tool schema attribute modify gecos --searchflags=8 </span><span>/usr/local/samba/bin/samba-tool schema attribute modify loginShell --searchflags=8 </span><span>/usr/local/samba/bin/samba-tool schema attribute modify home-directory --searchflags=24 </span></code></pre> <h2 id="ad-hardening">AD Hardening</h2> <p>We want to harden a few default settings that could be considered insecure. First, let's stop &quot;any user from being able to domain join machines&quot;.</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>/usr/local/samba/bin/samba-tool domain settings account_machine_join_quota 0 -H ldaps://localhost --simple-bind-dn=&#39;administrator@adt.blackhats.net.au&#39; </span></code></pre> <p>Now let's disable the Guest account</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>/usr/local/samba/bin/samba-tool user disable Guest -H ldaps://localhost --simple-bind-dn=&#39;administrator@adt.blackhats.net.au&#39; </span></code></pre> <p>I plan to write a more complete samba-tool extension for auditing these and more options, so stay tuned!</p> <h2 id="sssd-configuration">SSSD configuration</h2> <p>Now that our directory service is configured, we need to configure our clients to utilise it correctly.</p> <p>Here is my SSSD configuration, that supports sshPublicKey distribution, userCertificate authentication on workstations and SID -&gt; uid mapping. In the future I want to explore sudo rules in LDAP with AD, and maybe even HBAC rules rather than GPO.</p> <p>Please refer to my other blog posts on configuration of the userCertificates and sshKey distribution.</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>[domain/blackhats.net.au] </span><span>ignore_group_members = False </span><span> </span><span>debug_level=3 </span><span># There is a bug in SSSD where this actually means &quot;ipv6 only&quot;. </span><span># lookup_family_order=ipv6_first </span><span>cache_credentials = True </span><span>id_provider = ldap </span><span>auth_provider = ldap </span><span>access_provider = ldap </span><span>chpass_provider = ldap </span><span>ldap_search_base = dc=blackhats,dc=net,dc=au </span><span> </span><span># This prevents an infinite referral loop. </span><span>ldap_referrals = False </span><span>ldap_id_mapping = True </span><span>ldap_schema = ad </span><span># Rather that being in domain users group, create a user private group </span><span># automatically on login. </span><span># This is very important as a security setting on unix!!! </span><span># See this bug if it doesn&#39;t work correctly. </span><span># https://pagure.io/SSSD/sssd/issue/3723 </span><span>auto_private_groups = true </span><span> </span><span>ldap_uri = ldaps://ad.blackhats.net.au </span><span>ldap_tls_reqcert = demand </span><span>ldap_tls_cacert = /etc/pki/tls/certs/bh_ldap.crt </span><span> </span><span># Workstation access </span><span>ldap_access_filter = (memberOf=CN=Workstation Operators,CN=Users,DC=blackhats,DC=net,DC=au) </span><span> </span><span>ldap_user_member_of = memberof </span><span>ldap_user_gecos = cn </span><span>ldap_user_uuid = objectGUID </span><span>ldap_group_uuid = objectGUID </span><span># This is really important as it allows SSSD to respect nsAccountLock </span><span>ldap_account_expire_policy = ad </span><span>ldap_access_order = filter, expire </span><span># Setup for ssh keys </span><span>ldap_user_ssh_public_key = sshPublicKey </span><span># This does not require ;binary tag with AD. </span><span>ldap_user_certificate = userCertificate </span><span># This is required for the homeDirectory to be looked up in the sssd schema </span><span>ldap_user_home_directory = homeDirectory </span><span> </span><span> </span><span>[sssd] </span><span>services = nss, pam, ssh, sudo </span><span>config_file_version = 2 </span><span>certificate_verification = no_verification </span><span> </span><span>domains = blackhats.net.au </span><span>[nss] </span><span>homedir_substring = /home </span><span> </span><span>[pam] </span><span>pam_cert_auth = True </span><span> </span><span>[sudo] </span><span> </span><span>[autofs] </span><span> </span><span>[ssh] </span><span> </span><span>[pac] </span><span> </span><span>[ifp] </span></code></pre> <h2 id="conclusion">Conclusion</h2> <p>With these simple changes we can easily make samba 4 able to perform the roles of other unix focused LDAP servers. This allows stateless clients, secure ssh key authentication, certificate authentication and more.</p> <p>Some future goals to improve this include:</p> <ul> <li>Ship samba 4 with schema templates that can be used</li> <li>Schema querying (what objectclass takes this attribute?)</li> <li>Group editing (same as samba-tool user edit)</li> <li>Security auditing tools</li> <li>user/group modification commands</li> <li>Refactor and improve the cli tools python to be api driven - move the logic from netcmd into samdb so that samdb can be an API that python can consume easier. Prevent duplication of logic.</li> </ul> <p>The goal is so that an admin never has to see an LDIF ever again.</p> Smartcards and You - How To Make Them Work on Fedora/RHEL Tue, 27 Feb 2018 00:00:00 +0000 Unknown https://fy.blackhats.net.au/blog/2018-02-27-smartcards-and-you-how-to-make-them-work-on-fedora-rhel/ https://fy.blackhats.net.au/blog/2018-02-27-smartcards-and-you-how-to-make-them-work-on-fedora-rhel/ <h1 id="smartcards-and-you-how-to-make-them-work-on-fedora-rhel">Smartcards and You - How To Make Them Work on Fedora/RHEL</h1> <p>Smartcards are a great way to authenticate users. They have a device (something you have) and a pin (something you know). They prevent password transmission, use strong crypto and they even come in a variety of formats. From your &quot;card&quot; shapes to yubikeys.</p> <p>So why aren't they used more? It's the classic issue of usability - the setup for them is undocumented, complex, and hard to discover. Today I hope to change this.</p> <h2 id="the-goal">The Goal</h2> <p>To authenticate a user with a smartcard to a physical linux system, backed onto LDAP. The public cert in LDAP is validated, as is the chain to the CA.</p> <h2 id="you-will-need">You Will Need</h2> <ul> <li>LDAP (<a href="http://www.port389.org/">here is one I prepared earlier</a>)</li> <li>One Linux - Fedora 27 or RHEL 7 work best</li> <li>A smartcard (yubikey 4/nano works)</li> </ul> <p>I'll be focusing on the yubikey because that's what I own.</p> <h2 id="preparing-the-smartcard">Preparing the Smartcard</h2> <p>First we need to make the smartcard hold our certificate. Because of a crypto issue in yubikey firmware, it's best to generate certificates for these externally.</p> <p>I've documented this before in another post, but for accesibility here it is again.</p> <p>Create an NSS DB, and generate a certificate signing request:</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>certutil -d . -N -f pwdfile.txt </span><span>certutil -d . -R -a -o user.csr -f pwdfile.txt -g 4096 -Z SHA256 -v 24 \ </span><span>--keyUsage digitalSignature,nonRepudiation,keyEncipherment,dataEncipherment --nsCertType sslClient --extKeyUsage clientAuth \ </span><span>-s &quot;CN=username,O=Testing,L=example,ST=Queensland,C=AU&quot; </span></code></pre> <p>Once the request is signed, and your certificate is in &quot;user.crt&quot;, import this to the database.</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>certutil -A -d . -f pwdfile.txt -i user.crt -a -n TLS -t &quot;,,&quot; </span><span>certutil -A -d . -f pwdfile.txt -i ca.crt -a -n TLS -t &quot;CT,,&quot; </span></code></pre> <p>Now export that as a p12 bundle for the yubikey to import.</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>pk12util -o user.p12 -d . -k pwdfile.txt -n TLS </span></code></pre> <p>Now import this to the yubikey - remember to use slot 9a this time! As well make sure you set the touch policy NOW, because you can't change it later!</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>yubico-piv-tool -s9a -i user.p12 -K PKCS12 -aimport-key -aimport-certificate -k --touch-policy=always </span></code></pre> <h2 id="setting-up-your-ldap-user">Setting up your LDAP user</h2> <p>First setup your system to work with LDAP via SSSD. You've done that? Good! Now it's time to get our user ready.</p> <p>Take our user.crt and convert it to DER:</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>openssl x509 -inform PEM -outform DER -in user.crt -out user.der </span></code></pre> <p>Now you need to transform that into something that LDAP can understand. In the future I'll be adding a tool to 389-ds to make this &quot;automatic&quot;, but for now you can use python:</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>python3 </span><span>&gt;&gt;&gt; import base64 </span><span>&gt;&gt;&gt; with open(&#39;user.der&#39;, &#39;r&#39;) as f: </span><span>&gt;&gt;&gt; print(base64.b64encode(f.read)) </span></code></pre> <p>That should output a long base64 string on one line. Add this to your ldap user with ldapvi:</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>uid=william,ou=People,dc=... </span><span>userCertificate;binary:: &lt;BASE64&gt; </span></code></pre> <p>Note that ';binary' tag has an important meaning here for certificate data, and the '::' tells ldap that this is b64 encoded, so it will decode on addition.</p> <h2 id="setting-up-the-system">Setting up the system</h2> <p>Now that you have done that, you need to teach SSSD how to intepret that attribute.</p> <p>In your various SSSD sections you'll need to make the following changes:</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>[domain/LDAP] </span><span>auth_provider = ldap </span><span>ldap_user_certificate = userCertificate;binary </span><span> </span><span>[sssd] </span><span># This controls OCSP checks, you probably want this enabled! </span><span># certificate_verification = no_verification </span><span> </span><span>[pam] </span><span>pam_cert_auth = True </span></code></pre> <p>Now the TRICK is letting SSSD know to use certificates. You need to run:</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>sudo touch /var/lib/sss/pubconf/pam_preauth_available </span></code></pre> <p>With out this, SSSD won't even try to process CCID authentication!</p> <p>Add your ca.crt to the system trusted CA store for SSSD to verify:</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>certutil -A -d /etc/pki/nssdb -i ca.crt -n USER_CA -t &quot;CT,,&quot; </span></code></pre> <p>Add coolkey to the database so it can find smartcards:</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>modutil -dbdir /etc/pki/nssdb -add &quot;coolkey&quot; -libfile /usr/lib64/libcoolkeypk11.so </span></code></pre> <p>Check that SSSD can find the certs now:</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span># sudo /usr/libexec/sssd/p11_child --pre --nssdb=/etc/pki/nssdb </span><span>PIN for william </span><span>william </span><span>/usr/lib64/libcoolkeypk11.so </span><span>0001 </span><span>CAC ID Certificate </span></code></pre> <p>If you get no output here you are missing something! If this doesn't work, nothing will!</p> <p>Finally, you need to tweak PAM to make sure that pam_unix isn't getting in the way. I use the following configuration.</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>auth required pam_env.so </span><span># This skips pam_unix if the given uid is not local (IE it&#39;s from SSSD) </span><span>auth [default=1 ignore=ignore success=ok] pam_localuser.so </span><span>auth sufficient pam_unix.so nullok try_first_pass </span><span>auth requisite pam_succeed_if.so uid &gt;= 1000 quiet_success </span><span>auth sufficient pam_sss.so prompt_always ignore_unknown_user </span><span>auth required pam_deny.so </span><span> </span><span>account required pam_unix.so </span><span>account sufficient pam_localuser.so </span><span>account sufficient pam_succeed_if.so uid &lt; 1000 quiet </span><span>account [default=bad success=ok user_unknown=ignore] pam_sss.so </span><span>account required pam_permit.so </span><span> </span><span>password requisite pam_pwquality.so try_first_pass local_users_only retry=3 authtok_type= </span><span>password sufficient pam_unix.so sha512 shadow try_first_pass use_authtok </span><span>password sufficient pam_sss.so use_authtok </span><span>password required pam_deny.so </span><span> </span><span>session optional pam_keyinit.so revoke </span><span>session required pam_limits.so </span><span>-session optional pam_systemd.so </span><span>session [success=1 default=ignore] pam_succeed_if.so service in crond quiet use_uid </span><span>session required pam_unix.so </span><span>session optional pam_sss.so </span></code></pre> <p>That's it! Restart SSSD, and you should be good to go.</p> <p>Finally, you may find SELinux isn't allowing authentication. This is really sad that smartcards don't work with SELinux out of the box and I have raised a number of bugs, but check this just in case.</p> <p>Happy authentication!</p> Using b43 firmware on Fedora Atomic Workstation Sat, 23 Dec 2017 00:00:00 +0000 Unknown https://fy.blackhats.net.au/blog/2017-12-23-using-b43-firmware-on-fedora-atomic-workstation/ https://fy.blackhats.net.au/blog/2017-12-23-using-b43-firmware-on-fedora-atomic-workstation/ <h1 id="using-b43-firmware-on-fedora-atomic-workstation">Using b43 firmware on Fedora Atomic Workstation</h1> <p>My Macbook Pro has a broadcom b43 wireless chipset. This is notorious for being one of the most annoying wireless adapters on linux. When you first install Fedora you don't even see &quot;wifi&quot; as an option, and unless you poke around in dmesg, you won't find how to enable b43 to work on your platform.</p> <h2 id="b43">b43</h2> <p>The b43 driver requires proprietary firmware to be loaded else the wifi chip will not run. There are a number of steps for this process found on the linux wireless page . You'll note that one of the steps is:</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>export FIRMWARE_INSTALL_DIR=&quot;/lib/firmware&quot; </span><span>... </span><span>sudo b43-fwcutter -w &quot;$FIRMWARE_INSTALL_DIR&quot; broadcom-wl-5.100.138/linux/wl_apsta.o </span></code></pre> <p>So we need to be able to write and extract our firmware to /usr/lib/firmware, and then reboot and out wifi works.</p> <h2 id="fedora-atomic-workstation">Fedora Atomic Workstation</h2> <p>Atomic WS is similar to atomic server, that it's a read-only ostree based deployment of fedora. This comes with a number of unique challenges and quirks but for this issue:</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>sudo touch /usr/lib/firmware/test </span><span>/bin/touch: cannot touch &#39;/usr/lib/firmware/test&#39;: Read-only file system </span></code></pre> <p>So we can't extract our firmware!</p> <p>Normally linux also supports reading from /usr/local/lib/firmware (which on atomic IS writeable ...) but for some reason fedora doesn't allow this path.</p> <h2 id="solution-layered-rpms">Solution: Layered RPMs</h2> <p>Atomic has support for &quot;rpm layering&quot;. Ontop of the ostree image (which is composed of rpms) you can supply a supplemental list of packages that are &quot;installed&quot; at rpm-ostree update time.</p> <p>This way you still have an atomic base platform, with read-only behaviours, but you gain the ability to customise your system. To achive it, it must be possible to write to locations in /usr during rpm install.</p> <h2 id="new-method-install-rpmfusion-tainted">New method - install rpmfusion tainted</h2> <p>As I have now learnt, the b43 data is provided as part of rpmfusion nonfree. To enable this, you need to access the tainted repo. I a file such as &quot;/etc/yum.repos.d/rpmfusion-nonfree-tainted.repo&quot; add the content:</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>[rpmfusion-nonfree-tainted] </span><span>name=rpmfusion-nonfree-tainted </span><span>baseurl=https://download1.rpmfusion.org/nonfree/fedora/tainted/$releasever/$basearch/ </span><span>enabled=1 </span><span>gpgcheck=1 </span><span>type=rpm </span></code></pre> <p>Now, you should be able to run:</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>atomic host install b43-firmware </span><span>reboot </span></code></pre> <p>You should have a working wifi chipset!</p> <h2 id="custom-rpm-old-method">Custom RPM - old method</h2> <p>This means our problem has a simple solution: Create a b43 rpm package. Note, that you can make this for yourself privately, but you can't distribute it for legal reasons.</p> <p>Get setup on atomic to build the packages:</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>rpm-ostree install rpm-build createrepo </span><span>reboot </span></code></pre> <p>RPM specfile:</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>%define debug_package %{nil} </span><span>Summary: Allow b43 fw to install on ostree installs due to bz1512452 </span><span>Name: b43-fw </span><span>Version: 1.0.0 </span><span>Release: 1 </span><span>License: Proprietary, DO NOT DISTRIBUTE BINARY FORMS </span><span>URL: http://linuxwireless.sipsolutions.net/en/users/Drivers/b43/ </span><span>Group: System Environment/Kernel </span><span> </span><span>BuildRequires: b43-fwcutter </span><span> </span><span>Source0: http://www.lwfinger.com/b43-firmware/broadcom-wl-5.100.138.tar.bz2 </span><span> </span><span>%description </span><span>Broadcom firmware for b43 chips. </span><span> </span><span>%prep </span><span>%setup -q -n broadcom-wl-5.100.138 </span><span> </span><span>%build </span><span>true </span><span> </span><span>%install </span><span>pwd </span><span>mkdir -p %{buildroot}/usr/lib/firmware </span><span>b43-fwcutter -w %{buildroot}/usr/lib/firmware linux/wl_apsta.o </span><span> </span><span>%files </span><span>%defattr(-,root,root,-) </span><span>%dir %{_prefix}/lib/firmware/b43 </span><span>%{_prefix}/lib/firmware/b43/* </span><span> </span><span>%changelog </span><span>* Fri Dec 22 2017 William Brown &lt;william at blackhats.net.au&gt; - 1.0.0 </span><span>- Initial version </span></code></pre> <p>Now you can put this into a folder like so:</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>mkdir -p ~/rpmbuild/{SPECS,SOURCES} </span><span>&lt;editor&gt; ~/rpmbuild/SPECS/b43-fw.spec </span><span>wget -O ~/rpmbuild/SOURCES/broadcom-wl-5.100.138.tar.bz2 http://www.lwfinger.com/b43-firmware/broadcom-wl-5.100.138.tar.bz2 </span></code></pre> <p>We are now ready to build!</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>rpmbuild -bb ~/rpmbuild/SPECS/b43-fw.spec </span><span>createrepo ~/rpmbuild/RPMS/x86_64/ </span></code></pre> <p>Finally, we can install this. Create a yum repos file:</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>[local-rpms] </span><span>name=local-rpms </span><span>baseurl=file:///home/&lt;YOUR USERNAME HERE&gt;/rpmbuild/RPMS/x86_64 </span><span>enabled=1 </span><span>gpgcheck=0 </span><span>type=rpm </span><span> </span><span>rpm-ostree install b43-fw </span></code></pre> <p>Now reboot and enjoy wifi on your Fedora Atomic Macbook Pro!</p> Creating yubikey SSH and TLS certificates Sat, 11 Nov 2017 00:00:00 +0000 Unknown https://fy.blackhats.net.au/blog/2017-11-11-creating-yubikey-ssh-and-tls-certificates/ https://fy.blackhats.net.au/blog/2017-11-11-creating-yubikey-ssh-and-tls-certificates/ <h1 id="creating-yubikey-ssh-and-tls-certificates">Creating yubikey SSH and TLS certificates</h1> <p>Recently yubikeys were shown to have a hardware flaw in the way the generated private keys. This affects the use of them to provide PIV identies or SSH keys.</p> <p>However, you can generate the keys externally, and load them to the key to prevent this issue.</p> <h2 id="ssh">SSH</h2> <p>First, we'll create a new NSS DB on an airgapped secure machine (with disk encryption or in memory storage!)</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>certutil -N -d . -f pwdfile.txt </span></code></pre> <p>Now into this, we'll create a self-signed cert valid for 10 years.</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>certutil -S -f pwdfile.txt -d . -t &quot;C,,&quot; -x -n &quot;SSH&quot; -g 2048 -s &quot;cn=william,O=ssh,L=Brisbane,ST=Queensland,C=AU&quot; -v 120 </span></code></pre> <p>We export this now to PKCS12 for our key to import.</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>pk12util -o ssh.p12 -d . -k pwdfile.txt -n SSH </span></code></pre> <p>Next we import the key and cert to the hardware in slot 9c</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>yubico-piv-tool -s9c -i ssh.p12 -K PKCS12 -aimport-key -aimport-certificate -k </span></code></pre> <p>Finally, we can display the ssh-key from the token.</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>ssh-keygen -D /usr/lib64/opensc-pkcs11.so -e </span></code></pre> <p>Note, we can make this always used by ssh client by adding the following into .ssh/config:</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>PKCS11Provider /usr/lib64/opensc-pkcs11.so </span></code></pre> <h2 id="tls-identities">TLS Identities</h2> <p>The process is almost identical for user certificates.</p> <p>First, create the request:</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>certutil -d . -R -a -o user.csr -f pwdfile.txt -g 4096 -Z SHA256 -v 24 \ </span><span>--keyUsage digitalSignature,nonRepudiation,keyEncipherment,dataEncipherment --nsCertType sslClient --extKeyUsage clientAuth \ </span><span>-s &quot;CN=username,O=Testing,L=example,ST=Queensland,C=AU&quot; </span></code></pre> <p>Once the request is signed, we should have a user.crt back. Import that to our database:</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>certutil -A -d . -f pwdfile.txt -i user.crt -a -n TLS -t &quot;,,&quot; </span></code></pre> <p>Import our CA certificate also. Next export this to p12:</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>pk12util -o user.p12 -d . -k pwdfile.txt -n TLS </span></code></pre> <p>Now import this to the yubikey - remember to use slot 9a this time!</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>yubico-piv-tool -s9a -i user.p12 -K PKCS12 -aimport-key -aimport-certificate -k --touch-policy=always </span></code></pre> <p>Done!</p> What's the problem with NUMA anyway? Tue, 07 Nov 2017 00:00:00 +0000 Unknown https://fy.blackhats.net.au/blog/2017-11-07-what-s-the-problem-with-numa-anyway/ https://fy.blackhats.net.au/blog/2017-11-07-what-s-the-problem-with-numa-anyway/ <h1 id="what-s-the-problem-with-numa-anyway">What's the problem with NUMA anyway?</h1> <h2 id="what-is-numa">What is NUMA?</h2> <p>Non-Uniform Memory Architecture is a method of seperating ram and memory management units to be associated with CPU sockets. The reason for this is performance - if multiple sockets shared a MMU, they will cause each other to block, delaying your CPU.</p> <p>To improve this, each NUMA region has it's own MMU and RAM associated. If a CPU can access it's local MMU and RAM, this is very fast, and does <em>not</em> prevent another CPU from accessing it's own. For example:</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>CPU 0 &lt;-- QPI --&gt; CPU 1 </span><span> | | </span><span> v v </span><span>MMU 0 MMU 1 </span><span> | | </span><span> v v </span><span>RAM 1 RAM 2 </span></code></pre> <p>For example, on the following system, we can see 1 numa region:</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span># numactl --hardware </span><span>available: 1 nodes (0) </span><span>node 0 cpus: 0 1 2 3 </span><span>node 0 size: 12188 MB </span><span>node 0 free: 458 MB </span><span>node distances: </span><span>node 0 </span><span> 0: 10 </span></code></pre> <p>On this system, we can see two:</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span># numactl --hardware </span><span>available: 2 nodes (0-1) </span><span>node 0 cpus: 0 1 2 3 4 5 6 7 8 9 10 11 24 25 26 27 28 29 30 31 32 33 34 35 </span><span>node 0 size: 32733 MB </span><span>node 0 free: 245 MB </span><span>node 1 cpus: 12 13 14 15 16 17 18 19 20 21 22 23 36 37 38 39 40 41 42 43 44 45 46 47 </span><span>node 1 size: 32767 MB </span><span>node 1 free: 22793 MB </span><span>node distances: </span><span>node 0 1 </span><span> 0: 10 20 </span><span> 1: 20 10 </span></code></pre> <p>This means that on the second system there is 32GB of ram per NUMA region which is accessible, but the system has total 64GB.</p> <h2 id="the-problem">The problem</h2> <p>The problem arises when a process running on NUMA region 0 has to access memory from another NUMA region. Because there is no direct connection between CPU 0 and RAM 1, we must communicate with our neighbour CPU 1 to do this for us. IE:</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>CPU 0 --&gt; CPU 1 --&gt; MMU 1 --&gt; RAM 1 </span></code></pre> <p>Not only do we pay a time delay price for the QPI communication between CPU 0 and CPU 1, but now CPU 1's processes are waiting on the MMU 1 because we are retrieving memory on behalf of CPU 0. This is very slow (and can be seen by the node distances in the numactl --hardware output).</p> <h2 id="today-s-work-around">Today's work around</h2> <p>The work around today is to limit your Directory Server instance to a single NUMA region. So for our example above, we would limit the instance to NUMA region 0 or 1, and treat the instance as though it only has access to 32GB of local memory.</p> <p>It's possible to run two instances of DS on a single server, pinning them to their own regions and using replication between them to provide synchronisation. You'll need a load balancer to fix up the TCP port changes, or you need multiple addresses on the system for listening.</p> <h2 id="the-future">The future</h2> <p>In the future, we'll be adding support for better copy-on-write techniques that allow the cores to better cache content after a QPI negotiation - but we still have to pay the transit cost. We can minimise this as much as possible, but there is no way today to avoid this penalty. To use all your hardware on a single instance, there will always be a NUMA cost somewhere.</p> <p>The best solution is as above: run an instance per NUMA region, and internally provide replication for them. Perhaps we'll support an automatic configuration of this in the future.</p> GSoC 2017 - Mentor Report from 389 Project Thu, 24 Aug 2017 00:00:00 +0000 Unknown https://fy.blackhats.net.au/blog/2017-08-24-gsoc-2017-mentor-report-from-389-project/ https://fy.blackhats.net.au/blog/2017-08-24-gsoc-2017-mentor-report-from-389-project/ <h1 id="gsoc-2017-mentor-report-from-389-project">GSoC 2017 - Mentor Report from 389 Project</h1> <p>This year I have had the pleasure of being a mentor for the Google Summer of Code program, as part of the Fedora Project organisation. I was representing the <a href="http://www.port389.org/">389 Directory Server Project</a> and offered students the oppurtunity to <a href="https://fedoraproject.org/wiki/Summer_coding_ideas_for_2017#389_Directory_Server:_developing_administrative_tools">work on our command line tools</a> written in python.</p> <h2 id="applications">Applications</h2> <p>From the start we have a large number of really talented students apply to the project. This was one of the hardest parts of the process was to choose a student, given that I wanted to mentor all of them. Sadly I only have so many hours in the day, so we chose Ilias, a student from Greece. What really stood out was his interest in learning about the project, and his desire to really be part of the community after the project concluded.</p> <h2 id="the-project">The project</h2> <p>The project was very deliberately &quot;loose&quot; in it's specification. Rather than giving Ilias a fixed goal of you will implement X, Y and Z, I chose to set a &quot;broad and vague&quot; task. Initially I asked him to investigate a single area of the code (the MemberOf plugin). As he investigated this, he started to learn more about the server, ask questions, and open doors for himself to the next tasks of the project. As these smaller questions and self discoveries stacked up, I found myself watching Ilias start to become a really complete developer, who could be called a true part of our community.</p> <p>Ilias' work was exceptional, and he has documented it in his <a href="https://iliaswrites.wordpress.com/2017/08/23/final-gsoc-2017-report/">final report here</a> .</p> <p>Since his work is complete, he is now free to work on any task that takes his interest, and he has picked a good one! He has now started to dive deep into the server internals, looking at part of our backend internals and how we dump databases from id2entry to various output formats.</p> <h2 id="what-next">What next?</h2> <p>I will be participating next year - Sadly, I think the python project oppurtunities may be more limited as we have to finish many of these tasks to release our new CLI toolset. This is almost a shame as the python components are a great place to start as they ease a new contributor into the broader concepts of LDAP and the project structure as a whole.</p> <p>Next year I really want to give this oppurtunity to an under-represented group in tech (female, poc, etc). I personally have been really inspired by Noriko and I hope to have the oppurtunity to pass on her lessons to another aspiring student. We need more engineers like her in the world, and I want to help create that future.</p> <h2 id="advice-for-future-mentors">Advice for future mentors</h2> <p>Mentoring is not for everyone. It's not a task which you can just send a couple of emails and be done every day.</p> <p>Mentoring is a process that requires engagement with the student, and communication and the relationship is key to this. What worked well was meeting early in the project, and working out what community worked best for us. We found that email questions and responses worked (given we are on nearly opposite sides of the Earth) worked well, along with irc conversations to help fix up any other questions. It would not be uncommon for me to spend at least 1 or 2 hours a day working through emails from Ilias and discussions on IRC.</p> <p>A really important aspect of this communication is how you do it. You have to balance positive communication and encouragement, along with critcism that is constructive and helpful. Empathy is a super important part of this equation.</p> <p>My number one piece of advice would be that you need to create an environment where questions are encouraged and welcome. You can never be dismissive of questions. If ever you dismiss a question as &quot;silly&quot; or &quot;dumb&quot;, you will hinder a student from wanting to ask more questions. If you can't answer the question immediately, send a response saying &quot;hey I know this is important, but I'm really busy, I'll answer you as soon as I can&quot;.</p> <p>Over time you can use these questions to help teach lessons for the student to make their own discoveries. For example, when Ilias would ask how something worked, I would send my response structured in the way I approached the problem. I would send back links to code, my thoughts, and how I arrived at the conclusion. This not only answered the question but gave a subtle lesson in how to research our codebase to arrive at your own solutions. After a few of these emails, I'm sure that Ilias has now become self sufficent in his research of the code base.</p> <p>Another valuable skill is that overtime you can help to build confidence through these questions. To start with Ilias would ask &quot;how to implement&quot; something, and I would answer. Over time, he would start to provide ideas on how to implement a solution, and I would say &quot;X is the right one&quot;. As time went on I started to answer his question with &quot;What do you think is the right solution and why?&quot;. These exchanges and justifications have (I hope) helped him to become more confident in his ideas, the presentation of them, and justification of his solutions. It's led to this <a href="https://lists.fedoraproject.org/archives/list/389-devel@lists.fedoraproject.org/thread/5VDPWUZ3E67UMEWVCATU6LEIQ5QGBGEM/">excellent exchange</a> on our mailing lists, where Ilias is discussing the solutions to a problem with the broader community, and working to a really great answer.</p> <h2 id="final-thoughts">Final thoughts</h2> <p>This has been a great experience for myself and Ilias, and I really look forward to helping another student next year. I'm sure that Ilias will go on to do great things, and I'm happy to have been part of his journey.</p> So you want to script gdb with python ... Fri, 04 Aug 2017 00:00:00 +0000 Unknown https://fy.blackhats.net.au/blog/2017-08-04-so-you-want-to-script-gdb-with-python/ https://fy.blackhats.net.au/blog/2017-08-04-so-you-want-to-script-gdb-with-python/ <h1 id="so-you-want-to-script-gdb-with-python">So you want to script gdb with python ...</h1> <p>Gdb provides a python scripting interface. However the documentation is highly technical and not at a level that is easily accessible.</p> <p>This post should read as a tutorial, to help you understand the interface and work toward creating your own python debuging tools to help make gdb usage somewhat &quot;less&quot; painful.</p> <h2 id="the-problem">The problem</h2> <p>I have created a problem program called &quot;naughty&quot;. You can find it <a href="../../../_static/gdb_py/naughty.c">here</a> .</p> <p>You can compile this with the following command:</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>gcc -g -lpthread -o naughty naughty.c </span></code></pre> <p>When you run this program, your screen should be filled with:</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>thread ... </span><span>thread ... </span><span>thread ... </span><span>thread ... </span><span>thread ... </span><span>thread ... </span></code></pre> <p>It looks like we have a bug! Now, we could easily see the issue if we looked at the C code, but that's not the point here - lets try to solve this with gdb.</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>gdb ./naughty </span><span>... </span><span>(gdb) run </span><span>... </span><span>[New Thread 0x7fffb9792700 (LWP 14467)] </span><span>... </span><span>thread ... </span></code></pre> <p>Uh oh! We have threads being created here. We need to find the problem thread. Lets look at all the threads backtraces then.</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>Thread 129 (Thread 0x7fffb3786700 (LWP 14616)): </span><span>#0 0x00007ffff7bc38eb in pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0 </span><span>#1 0x00000000004007bc in lazy_thread (arg=0x7fffffffdfb0) at naughty.c:19 </span><span>#2 0x00007ffff7bbd3a9 in start_thread () from /lib64/libpthread.so.0 </span><span>#3 0x00007ffff78e936f in clone () from /lib64/libc.so.6 </span><span> </span><span>Thread 128 (Thread 0x7fffb3f87700 (LWP 14615)): </span><span>#0 0x00007ffff7bc38eb in pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0 </span><span>#1 0x00000000004007bc in lazy_thread (arg=0x7fffffffdfb0) at naughty.c:19 </span><span>#2 0x00007ffff7bbd3a9 in start_thread () from /lib64/libpthread.so.0 </span><span>#3 0x00007ffff78e936f in clone () from /lib64/libc.so.6 </span><span> </span><span>Thread 127 (Thread 0x7fffb4788700 (LWP 14614)): </span><span>#0 0x00007ffff7bc38eb in pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0 </span><span>#1 0x00000000004007bc in lazy_thread (arg=0x7fffffffdfb0) at naughty.c:19 </span><span>#2 0x00007ffff7bbd3a9 in start_thread () from /lib64/libpthread.so.0 </span><span>#3 0x00007ffff78e936f in clone () from /lib64/libc.so.6 </span><span> </span><span>... </span></code></pre> <p>We have 129 threads! Anyone of them could be the problem. We could just read these traces forever, but that's a waste of time. Let's try and script this with python to make our lives a bit easier.</p> <h2 id="python-in-gdb">Python in gdb</h2> <p>Python in gdb works by bringing in a copy of the python and injecting a special &quot;gdb&quot; module into the python run time. You can <em>only</em> access the gdb module from within python if you are using gdb. You can not have this work from a standard interpretter session.</p> <p>We can access a dynamic python runtime from within gdb by simply calling python.</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>(gdb) python </span><span>&gt;print(&quot;hello world&quot;) </span><span>&gt;hello world </span><span>(gdb) </span></code></pre> <p>The python code only runs when you press Control D.</p> <p>Another way to run your script is to import them as &quot;new gdb commands&quot;. This is the most useful way to use python for gdb, but it does require some boilerplate to start.</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>import gdb </span><span> </span><span>class SimpleCommand(gdb.Command): </span><span> def __init__(self): </span><span> # This registers our class as &quot;simple_command&quot; </span><span> super(SimpleCommand, self).__init__(&quot;simple_command&quot;, gdb.COMMAND_DATA) </span><span> </span><span> def invoke(self, arg, from_tty): </span><span> # When we call &quot;simple_command&quot; from gdb, this is the method </span><span> # that will be called. </span><span> print(&quot;Hello from simple_command!&quot;) </span><span> </span><span># This registers our class to the gdb runtime at &quot;source&quot; time. </span><span>SimpleCommand() </span></code></pre> <p>We can run the command as follows:</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>(gdb) source debug_naughty.py </span><span>(gdb) simple_command </span><span>Hello from simple_command! </span><span>(gdb) </span></code></pre> <h2 id="solving-the-problem-with-python">Solving the problem with python</h2> <p>So we need a way to find the &quot;idle threads&quot;. We want to fold all the threads with the same frame signature into one, so that we can view anomalies.</p> <p>First, let's make a &quot;stackfold&quot; command, and get it to list the current program.</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>class StackFold(gdb.Command): </span><span>def __init__(self): </span><span> super(StackFold, self).__init__(&quot;stackfold&quot;, gdb.COMMAND_DATA) </span><span> </span><span>def invoke(self, arg, from_tty): </span><span> # An inferior is the &#39;currently running applications&#39;. In this case we only </span><span> # have one. </span><span> inferiors = gdb.inferiors() </span><span> for inferior in inferiors: </span><span> print(inferior) </span><span> print(dir(inferior)) </span><span> print(help(inferior)) </span><span> </span><span>StackFold() </span></code></pre> <p>To reload this in the gdb runtime, just run &quot;source debug_naughty.py&quot; again. Try running this: Note that we dumped a heap of output? Python has a neat trick that dir and help can both return strings for printing. This will help us to explore gdb's internals inside of our program.</p> <p>We can see from the inferiors that we have threads available for us to interact with:</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>class Inferior(builtins.object) </span><span> | GDB inferior object </span><span>... </span><span> | threads(...) </span><span> | Return all the threads of this inferior. </span></code></pre> <p>Given we want to fold the stacks from all our threads, we probably need to look at this! So lets get one thread from this, and have a look at it's help.</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>inferiors = gdb.inferiors() </span><span>for inferior in inferiors: </span><span> thread_iter = iter(inferior.threads()) </span><span> head_thread = next(thread_iter) </span><span> print(help(head_thread)) </span></code></pre> <p>Now we can run this by re-running &quot;source&quot; on our script, and calling stackfold again, we see help for our threads in the system.</p> <p>At this point it get's a little bit less obvious. Gdb's python integration relates closely to how a human would interact with gdb. In order to access the content of a thread, we need to change the gdb context to access the backtrace. If we were doing this by hand it would look like this:</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>(gdb) thread 121 </span><span>[Switching to thread 121 (Thread 0x7fffb778e700 (LWP 14608))] </span><span>#0 0x00007ffff7bc38eb in pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0 </span><span>(gdb) bt </span><span>#0 0x00007ffff7bc38eb in pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0 </span><span>#1 0x00000000004007bc in lazy_thread (arg=0x7fffffffdfb0) at naughty.c:19 </span><span>#2 0x00007ffff7bbd3a9 in start_thread () from /lib64/libpthread.so.0 </span><span>#3 0x00007ffff78e936f in clone () from /lib64/libc.so.6 </span></code></pre> <p>We need to emulate this behaviour with our python calls. We can swap to the thread's context with:</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>class InferiorThread(builtins.object) </span><span> | GDB thread object </span><span>... </span><span> | switch(...) </span><span> | switch () </span><span> | Makes this the GDB selected thread. </span></code></pre> <p>Then once we are in the context, we need to take a different approach to explore the stack frames. We need to explore the &quot;gdb&quot; modules raw context.</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>inferiors = gdb.inferiors() </span><span>for inferior in inferiors: </span><span> thread_iter = iter(inferior.threads()) </span><span> head_thread = next(thread_iter) </span><span> # Move our gdb context to the selected thread here. </span><span> head_thread.switch() </span><span> print(help(gdb)) </span></code></pre> <p>Now that we have selected our thread's context, we can start to explore here. gdb can do a lot within the selected context - as a result, the help output from this call is really large, but it's worth reading so you can understand what is possible to achieve. In our case we need to start to look at the stack frames.</p> <p>To look through the frames we need to tell gdb to rewind to the &quot;newest&quot; frame (ie, frame 0). We can then step down through progressively older frames until we exhaust. From this we can print a rudimentary trace:</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>head_thread.switch() </span><span> </span><span># Reset the gdb frame context to the &quot;latest&quot; frame. </span><span>gdb.newest_frame() </span><span># Now, work down the frames. </span><span>cur_frame = gdb.selected_frame() </span><span>while cur_frame is not None: </span><span> print(cur_frame.name()) </span><span> # get the next frame down .... </span><span> cur_frame = cur_frame.older() </span><span> </span><span>(gdb) stackfold </span><span>pthread_cond_wait@@GLIBC_2.3.2 </span><span>lazy_thread </span><span>start_thread </span><span>clone </span></code></pre> <p>Great! Now we just need some extra metadata from the thread to know what thread id it is so the user can go to the correct thread context. So lets display that too:</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>head_thread.switch() </span><span> </span><span># These are the OS pid references. </span><span>(tpid, lwpid, tid) = head_thread.ptid </span><span># This is the gdb thread number </span><span>gtid = head_thread.num </span><span>print(&quot;tpid %s, lwpid %s, tid %s, gtid %s&quot; % (tpid, lwpid, tid, gtid)) </span><span># Reset the gdb frame context to the &quot;latest&quot; frame. </span><span> </span><span>(gdb) stackfold </span><span>tpid 14485, lwpid 14616, tid 0, gtid 129 </span></code></pre> <p>At this point we have enough information to fold identical stacks. We'll iterate over every thread, and if we have seen the &quot;pattern&quot; before, we'll just add the gdb thread id to the list. If we haven't seen the pattern yet, we'll add it. The final command looks like:</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>def invoke(self, arg, from_tty): </span><span> # An inferior is the &#39;currently running applications&#39;. In this case we only </span><span> # have one. </span><span> stack_maps = {} </span><span> # This creates a dict where each element is keyed by backtrace. </span><span> # Then each backtrace contains an array of &quot;frames&quot; </span><span> # </span><span> inferiors = gdb.inferiors() </span><span> for inferior in inferiors: </span><span> for thread in inferior.threads(): </span><span> # Change to our threads context </span><span> thread.switch() </span><span> # Get the thread IDS </span><span> (tpid, lwpid, tid) = thread.ptid </span><span> gtid = thread.num </span><span> # Take a human readable copy of the backtrace, we&#39;ll need this for display later. </span><span> o = gdb.execute(&#39;bt&#39;, to_string=True) </span><span> # Build the backtrace for comparison </span><span> backtrace = [] </span><span> gdb.newest_frame() </span><span> cur_frame = gdb.selected_frame() </span><span> while cur_frame is not None: </span><span> backtrace.append(cur_frame.name()) </span><span> cur_frame = cur_frame.older() </span><span> # Now we have a backtrace like [&#39;pthread_cond_wait@@GLIBC_2.3.2&#39;, &#39;lazy_thread&#39;, &#39;start_thread&#39;, &#39;clone&#39;] </span><span> # dicts can&#39;t use lists as keys because they are non-hashable, so we turn this into a string. </span><span> # Remember, C functions can&#39;t have spaces in them ... </span><span> s_backtrace = &#39; &#39;.join(backtrace) </span><span> # Let&#39;s see if it exists in the stack_maps </span><span> if s_backtrace not in stack_maps: </span><span> stack_maps[s_backtrace] = [] </span><span> # Now lets add this thread to the map. </span><span> stack_maps[s_backtrace].append({&#39;gtid&#39;: gtid, &#39;tpid&#39; : tpid, &#39;bt&#39;: o} ) </span><span> # Now at this point we have a dict of traces, and each trace has a &quot;list&quot; of pids that match. Let&#39;s display them </span><span> for smap in stack_maps: </span><span> # Get our human readable form out. </span><span> o = stack_maps[smap][0][&#39;bt&#39;] </span><span> for t in stack_maps[smap]: </span><span> # For each thread we recorded </span><span> print(&quot;Thread %s (LWP %s))&quot; % (t[&#39;gtid&#39;], t[&#39;tpid&#39;])) </span><span> print(o) </span></code></pre> <p>Here is the final output.</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>(gdb) stackfold </span><span>Thread 129 (LWP 14485)) </span><span>Thread 128 (LWP 14485)) </span><span>Thread 127 (LWP 14485)) </span><span>... </span><span>Thread 10 (LWP 14485)) </span><span>Thread 9 (LWP 14485)) </span><span>Thread 8 (LWP 14485)) </span><span>Thread 7 (LWP 14485)) </span><span>Thread 6 (LWP 14485)) </span><span>Thread 5 (LWP 14485)) </span><span>Thread 4 (LWP 14485)) </span><span>Thread 3 (LWP 14485)) </span><span>#0 0x00007ffff7bc38eb in pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0 </span><span>#1 0x00000000004007bc in lazy_thread (arg=0x7fffffffdfb0) at naughty.c:19 </span><span>#2 0x00007ffff7bbd3a9 in start_thread () from /lib64/libpthread.so.0 </span><span>#3 0x00007ffff78e936f in clone () from /lib64/libc.so.6 </span><span> </span><span>Thread 2 (LWP 14485)) </span><span>#0 0x00007ffff78d835b in write () from /lib64/libc.so.6 </span><span>#1 0x00007ffff78524fd in _IO_new_file_write () from /lib64/libc.so.6 </span><span>#2 0x00007ffff7854271 in __GI__IO_do_write () from /lib64/libc.so.6 </span><span>#3 0x00007ffff7854723 in __GI__IO_file_overflow () from /lib64/libc.so.6 </span><span>#4 0x00007ffff7847fa2 in puts () from /lib64/libc.so.6 </span><span>#5 0x00000000004007e9 in naughty_thread (arg=0x0) at naughty.c:27 </span><span>#6 0x00007ffff7bbd3a9 in start_thread () from /lib64/libpthread.so.0 </span><span>#7 0x00007ffff78e936f in clone () from /lib64/libc.so.6 </span><span> </span><span>Thread 1 (LWP 14485)) </span><span>#0 0x00007ffff7bbe90d in pthread_join () from /lib64/libpthread.so.0 </span><span>#1 0x00000000004008d1 in main (argc=1, argv=0x7fffffffe508) at naughty.c:51 </span></code></pre> <p>With our stackfold command we can easily see that threads 129 through 3 have the same stack, and are idle. We can see that tread 1 is the main process waiting on the threads to join, and finally we can see that thread 2 is the culprit writing to our display.</p> <h2 id="my-solution">My solution</h2> <p>You can find my solution to this problem as a <a href="../../../_static/gdb_py/debug_naughty.py">reference implementation here</a> .</p> Time safety and Rust Wed, 12 Jul 2017 00:00:00 +0000 Unknown https://fy.blackhats.net.au/blog/2017-07-12-time-safety-and-rust/ https://fy.blackhats.net.au/blog/2017-07-12-time-safety-and-rust/ <h1 id="time-safety-and-rust">Time safety and Rust</h1> <p>Recently I have had the great fortune to work on <a href="https://pagure.io/389-ds-base/issue/49316">this ticket</a> . This was an issue that stemmed from an attempt to make clock performance faster. Previously, a call to time or clock_gettime would involve a context switch an a system call (think solaris etc). On linux we have VDSO instead, so we can easily just swap to the use of raw time calls.</p> <h2 id="the-problem">The problem</h2> <p>So what was the problem? And how did the engineers of the past try and solve it?</p> <p>DS heavily relies on time. As a result, we call time() a <em>lot</em> in the codebase. But this would mean context switches.</p> <p>So a wrapper was made called &quot;current_time()&quot;, which would cache a recent output of time(), and then provide that to the caller instead of making the costly context switch. So the code had the following:</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>static time_t currenttime; </span><span>static int currenttime_set = 0; </span><span> </span><span>time_t </span><span>poll_current_time() </span><span>{ </span><span> if ( !currenttime_set ) { </span><span> currenttime_set = 1; </span><span> } </span><span> </span><span> time( &amp;currenttime ); </span><span> return( currenttime ); </span><span>} </span><span> </span><span>time_t </span><span>current_time( void ) </span><span>{ </span><span> if ( currenttime_set ) { </span><span> return( currenttime ); </span><span> } else { </span><span> return( time( (time_t *)0 )); </span><span> } </span><span>} </span></code></pre> <p>In another thread, we would poll this every second to update the currenttime value:</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>void * </span><span>time_thread(void *nothing __attribute__((unused))) </span><span>{ </span><span> PRIntervalTime interval; </span><span> </span><span> interval = PR_SecondsToInterval(1); </span><span> </span><span> while(!time_shutdown) { </span><span> poll_current_time(); </span><span> csngen_update_time (); </span><span> DS_Sleep(interval); </span><span> } </span><span> </span><span> /*NOTREACHED*/ </span><span> return(NULL); </span><span>} </span></code></pre> <p><em>So what is the problem here</em></p> <p>Besides the fact that we may not poll accurately (meaning we miss seconds but always advance), this is not <em>thread safe</em>. The reason is that CPU's have register and buffers that may cache both stores and writes until a series of other operations (barriers + atomics) occur to flush back out to cache. This means the time polling thread could update the clock and unless the POLLING thread issues a lock or a barrier+atomic, there is <em>no guarantee the new value of currenttime will be seen in any other thread</em>. This means that the only way this worked was by luck, and no one noticing that time would jump about or often just be wrong.</p> <p>Clearly this is a broken design, but this is C - we can do anything.</p> <h2 id="what-if-this-was-rust">What if this was Rust?</h2> <p>Rust touts mulithread safety high on it's list. So lets try and recreate this in rust.</p> <p>First, the exact same way:</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>use std::time::{SystemTime, Duration}; </span><span>use std::thread; </span><span> </span><span> </span><span>static mut currenttime: Option&lt;SystemTime&gt; = None; </span><span> </span><span>fn read_thread() { </span><span> let interval = Duration::from_secs(1); </span><span> </span><span> for x in 0..10 { </span><span> thread::sleep(interval); </span><span> let c_time = currenttime.unwrap(); </span><span> println!(&quot;reading time {:?}&quot;, c_time); </span><span> } </span><span>} </span><span> </span><span>fn poll_thread() { </span><span> let interval = Duration::from_secs(1); </span><span> </span><span> for x in 0..10 { </span><span> currenttime = Some(SystemTime::now()); </span><span> println!(&quot;polling time&quot;); </span><span> thread::sleep(interval); </span><span> } </span><span>} </span><span> </span><span>fn main() { </span><span> let poll = thread::spawn(poll_thread); </span><span> let read = thread::spawn(read_thread); </span><span> read.join().unwrap(); </span><span> poll.join().unwrap(); </span><span>} </span></code></pre> <p><em>Rust will not compile this code</em>.</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>&gt; rustc clock.rs </span><span>error[E0133]: use of mutable static requires unsafe function or block </span><span> --&gt; clock.rs:13:22 </span><span> | </span><span>13 | let c_time = currenttime.unwrap(); </span><span> | ^^^^^^^^^^^ use of mutable static </span><span> </span><span>error[E0133]: use of mutable static requires unsafe function or block </span><span> --&gt; clock.rs:22:9 </span><span> | </span><span>22 | currenttime = Some(SystemTime::now()); </span><span> | ^^^^^^^^^^^ use of mutable static </span><span> </span><span>error: aborting due to 2 previous errors </span></code></pre> <p>Rust has told us that this action is <em>unsafe</em>, and that we shouldn't be modifying a global static like this.</p> <p>This alone is a great reason and demonstration of why we need a language like Rust instead of C - the compiler can tell us when actions are dangerous at compile time, rather than being allowed to sit in production code for years.</p> <p>For bonus marks, because Rust is stricter about types than C, we don't have issues like:</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>int c_time = time(); </span></code></pre> <p>Which is a 2038 problem in the making :)</p> indexed search performance for ds - the mystery of the and query Mon, 26 Jun 2017 00:00:00 +0000 Unknown https://fy.blackhats.net.au/blog/2017-06-26-indexed-search-performance-for-ds-the-mystery-of-the-and-query/ https://fy.blackhats.net.au/blog/2017-06-26-indexed-search-performance-for-ds-the-mystery-of-the-and-query/ <h1 id="indexed-search-performance-for-ds-the-mystery-of-the-and-query">indexed search performance for ds - the mystery of the and query</h1> <p>Directory Server is heavily based on set mathematics - one of the few topics I enjoyed during university. Our filters really boil down to set queries:</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>&amp;((attr=val1)(attr=val2)) </span></code></pre> <p>This filter describes the intersection of sets of objects containing &quot;attr=val1&quot; and &quot;attr=val2&quot;.</p> <p>One of the properties of sets is that operations on them are commutative - the sets to a union or intersection may be supplied in any order with the same results. As a result, these are equivalent:</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>&amp;(a)(b)(c) </span><span>&amp;(b)(a)(c) </span><span>&amp;(c)(b)(a) </span><span>&amp;(c)(a)(b) </span><span>... </span></code></pre> <p>In the past I noticed an odd behaviour: that the <em>order</em> of filter terms in an ldapsearch query would drastically change the performance of the search. For example:</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>&amp;(a)(b)(c) </span><span>&amp;(c)(b)(a) </span></code></pre> <p>The later query may significantly outperform the former - but 10% or greater. I have never understood the reason why though. I toyed with ideas of re-arranging queries in the optimise step to put the terms in a better order, but I didn't know what factors affected this behaviour.</p> <p>Over time I realised that if you put the &quot;more specific&quot; filters first over the general filters, you would see a performance increase.</p> <h2 id="what-was-going-on">What was going on?</h2> <p>Recently I was asked to investigate a full table scan issue with range queries. This led me into an exploration of our search internals, and yielded the answer to the issue above.</p> <p>Inside of directory server, our indexes are maintained as &quot;pre-baked&quot; searches. Rather than trying to search every object to see if a filter matches, our indexes contain a list of entries that match a term. For example:</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>uid=mark: 1, 2 </span><span>uid=william: 3 </span><span>uid=noriko: 4 </span></code></pre> <p>From each indexed term we construct an IDList, which is the set of entries matching some term.</p> <p>On a complex query we would need to intersect these. So the algorithm would iteratively apply this:</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>t1 = (a, b) </span><span>t2 = (c, t1) </span><span>t3 = (d, t2) </span><span>... </span></code></pre> <p>In addition, the intersection would allocate a new IDList to insert the results into.</p> <p>What would happen is that if your first terms were large, we would allocate large IDLists, and do many copies into it. This would also affect later filters as we would need to check large ID spaces to perform the final intersection.</p> <p>In the above example, consider a, b, c all have 10,000 candidates. This would mean t1, t2 is at least 10,000 IDs, and we need to do at least 20,000 comparisons. If d were only 3 candidates, this means that we then throw away the majority of work and allocations when we get to t3 = (d, t2).</p> <h2 id="what-is-the-fix">What is the fix?</h2> <p>We now wrap each term in an idl_set processing api. When we get the IDList from each AVA, we insert it to the idl_set. This tracks the &quot;minimum&quot; IDList, and begins our intersection from the smallest matching IDList. This means that we have the quickest reduction in set size, and results in the smallest possible IDList allocation for the results. In my tests I have seen up to 10% improvement on complex queries.</p> <p>For the example above, this means we procees d first, to reduce t1 to the smallest possible candidate set we can.</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>t1 = (d, a) </span><span>t2 = (b, t1) </span><span>t3 = (c, t2) </span><span>... </span></code></pre> <p>This means to create t2, t3, we will do an allocation that is bounded by the size of d (aka 3, rather than 10,000), we only need to perform fewer queries to reach this point.</p> <p>A benefit of this strategy is that it means if on the first operation we find t1 is empty set, we can return <em>immediately</em> because no other intersection will have an impact on the operation.</p> <h2 id="what-is-next">What is next?</h2> <p>I still have not improved union performance - this is still somewhat affected by the ordering of terms in a filter. However, I have a number of ideas related to either bitmask indexes or disjoin set structures that can be used to improve this performance.</p> <p>Stay tuned ....</p> TLS Authentication and FreeRADIUS Thu, 25 May 2017 00:00:00 +0000 Unknown https://fy.blackhats.net.au/blog/2017-05-25-tls-authentication-and-freeradius/ https://fy.blackhats.net.au/blog/2017-05-25-tls-authentication-and-freeradius/ <h1 id="tls-authentication-and-freeradius">TLS Authentication and FreeRADIUS</h1> <p>In a push to try and limit the amount of passwords sent on my network, I'm changing my wireless to use TLS certificates for authentication.</p> Kerberos - why the world moved on Tue, 23 May 2017 00:00:00 +0000 Unknown https://fy.blackhats.net.au/blog/2017-05-23-kerberos-why-the-world-moved-on/ https://fy.blackhats.net.au/blog/2017-05-23-kerberos-why-the-world-moved-on/ <h1 id="kerberos-why-the-world-moved-on">Kerberos - why the world moved on</h1> <p>For a long time I have tried to integrate and improve authentication technologies in my own environments. I have advocated the use of GSSAPI, IPA, AD, and others. However, the more I have learnt, the further I have seen the world moving away. I want to explore some of my personal experiences and views as to why this occured, and what we can do.</p> Custom OSTree images Mon, 22 May 2017 00:00:00 +0000 Unknown https://fy.blackhats.net.au/blog/2017-05-22-custom-ostree-images/ https://fy.blackhats.net.au/blog/2017-05-22-custom-ostree-images/ <h1 id="custom-ostree-images">Custom OSTree images</h1> <p><a href="https://www.projectatomic.io/">Project Atomic</a> is in my view, one of the most promising changes to come to linux distributions in a long time. It boasts the ability to atomicupgrade and alter your OS by maintaining A/B roots of the filesystem. It is currently focused on docker and k8s runtimes, but we can use atomic in other locations.</p> Your Code Has Impact Fri, 10 Mar 2017 00:00:00 +0000 Unknown https://fy.blackhats.net.au/blog/2017-03-10-your-code-has-impact/ https://fy.blackhats.net.au/blog/2017-03-10-your-code-has-impact/ <h1 id="your-code-has-impact">Your Code Has Impact</h1> <p>As an engineer, sometimes it's easy to forget <em>why</em> we are writing programs. Deep in a bug hunt, or designing a new feature it's really easy to focus so hard on these small things you forget the bigger picture. I've even been there and made this mistake.</p> CVE-2017-2591 - DoS via OOB heap read Wed, 22 Feb 2017 00:00:00 +0000 Unknown https://fy.blackhats.net.au/blog/2017-02-22-cve-2017-2591-dos-via-oob-heap-read/ https://fy.blackhats.net.au/blog/2017-02-22-cve-2017-2591-dos-via-oob-heap-read/ <h1 id="cve-2017-2591-dos-via-oob-heap-read">CVE-2017-2591 - DoS via OOB heap read</h1> <p>On 18 of Jan 2017, the following <a href="http://seclists.org/oss-sec/2017/q1/129">email found it's way to my notifications</a> .</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>This is to disclose the following CVE: </span><span> </span><span>CVE-2017-2591 389 Directory Server: DoS via OOB heap read </span><span> </span><span>Description : </span><span> </span><span>The &quot;attribute uniqueness&quot; plugin did not properly NULL-terminate an array </span><span>when building up its configuration, if a so called &#39;old-style&#39; </span><span>configuration, was being used (Using nsslapd-pluginarg&lt;X&gt; parameters) . </span><span> </span><span>A attacker, authenticated, but possibly also unauthenticated, could </span><span>possibly force the plugin to read beyond allocated memory and trigger a </span><span>segfault. </span><span> </span><span>The crash could also possibly be triggered accidentally. </span><span> </span><span>Upstream patch : </span><span>https://fedorahosted.org/389/changeset/ffda694dd622b31277da07be76d3469fad86150f/ </span><span>Affected versions : from 1.3.4.0 </span><span> </span><span>Fixed version : 1.3.6 </span><span> </span><span>Impact: Low </span><span>CVSS3 scoring : 3.7 -- CVSS:3.0/AV:N/AC:H/PR:N/UI:N/S:U/C:N/I:N/A:L </span><span> </span><span>Upstream bug report : https://fedorahosted.org/389/ticket/48986 </span></code></pre> <p>So I decided to pull this apart: Given I found the issue and wrote the fix, I didn't deem it security worthy, so why was a CVE raised?</p> LCA2017 - Getting Into the Rusty Bucket Mon, 23 Jan 2017 00:00:00 +0000 Unknown https://fy.blackhats.net.au/blog/2017-01-23-lca2017-getting-into-the-rusty-bucket/ https://fy.blackhats.net.au/blog/2017-01-23-lca2017-getting-into-the-rusty-bucket/ <h1 id="lca2017-getting-into-the-rusty-bucket">LCA2017 - Getting Into the Rusty Bucket</h1> <p>I spoke at <a href="http://lca2017.org">Linux Conf Australia 2017</a> recently. I presented techniques and lessons about integrating Rust with existing C code bases. This is related to my work on Directory Server.</p> <p>The recording of the talk can be found on <a href="https://www.youtube.com/watch?v=AWnza5JX7jQ">youtube</a> and on the <a href="http://mirror.linux.org.au/pub/linux.conf.au/2017/">Linux Australia Mirror</a> .</p> <p>You can find the git repository for the project <a href="https://github.com/Firstyear/ds_rust">on github</a> .</p> <p>The slides can be viewed on <a href="http://redhat.slides.com/wibrown/rusty-bucket?token=oPNS4Ilp">slides.com</a> .</p> <p>I have already had a lot of feedback on improvements to make to this system including the use of struct pointers instead of c_void, and the use of bindgen in certain places.</p> The next year of Directory Server Mon, 23 Jan 2017 00:00:00 +0000 Unknown https://fy.blackhats.net.au/blog/2017-01-23-the-next-year-of-directory-server/ https://fy.blackhats.net.au/blog/2017-01-23-the-next-year-of-directory-server/ <h1 id="the-next-year-of-directory-server">The next year of Directory Server</h1> <p>Last year I wrote a post about the vision behind Directory Server and what I wanted to achieve in the project personally. My key aims were:</p> <ul> <li>We need to modernise our tooling, and installers.</li> <li>Setting up replication groups and masters needs to be simpler.</li> <li>We need to get away from long lived static masters.</li> <li>During updates, we need to start to enable smarter choices by default.</li> <li>Out of the box we need smarter settings.</li> <li>Web Based authentication</li> </ul> Usability of software: The challenges facing projects Mon, 23 Jan 2017 00:00:00 +0000 Unknown https://fy.blackhats.net.au/blog/2017-01-23-usability-of-software-the-challenges-facing-projects/ https://fy.blackhats.net.au/blog/2017-01-23-usability-of-software-the-challenges-facing-projects/ <h1 id="usability-of-software-the-challenges-facing-projects">Usability of software: The challenges facing projects</h1> <p>I have always desired the usability of software like Directory Server to improve. As a former system administrator, usabilty and documentation are very important for me. Improvements to usability can eliminate load on documentation, support services and more.</p> <p>Consider a microwave. No one reads the user manual. They unbox it, plug it in, and turn it on. You punch in a time and expect it to &quot;make cold things hot&quot;. You only consult the manual if it blows up.</p> <p>Many of these principles are rooted in the field of design. Design is an important and often over looked part of software development - All the way from the design of an API to the configuration, and even the user interface of software.</p> State of the 389 ds port, 2017 Fri, 06 Jan 2017 00:00:00 +0000 Unknown https://fy.blackhats.net.au/blog/2017-01-06-state-of-the-389-ds-port-2017/ https://fy.blackhats.net.au/blog/2017-01-06-state-of-the-389-ds-port-2017/ <h1 id="state-of-the-389-ds-port-2017">State of the 389 ds port, 2017</h1> <p>Previously I have written about my efforts to port 389 ds to FreeBSD.</p> <p>A great deal of progress has been made in the last few weeks (owing to my taking time off work).</p> <p>I have now ported nunc-stans to freebsd, which is important as it's our new connection management system. It has an issue with long lived events, but I will resolve this soon.</p> <p>The majority of patches for 389 ds have merged, with a single patch remaining to be reviewed.</p> <p>Finally, I have build a freebsd makefile for the <a href="https://github.com/Firstyear/ds-devel-root/blob/master/Makefile.fbsd">devel root</a> to make it easier for people to install and test from source.</p> <p>Once the freebsd nunc-stans and final DS patch are accepted, I'll be able to start building the portfiles.</p> Openshift cluster administration Mon, 02 Jan 2017 00:00:00 +0000 Unknown https://fy.blackhats.net.au/blog/2017-01-02-openshift-cluster-administration/ https://fy.blackhats.net.au/blog/2017-01-02-openshift-cluster-administration/ <h1 id="openshift-cluster-administration">Openshift cluster administration</h1> <p>Over the last 6 months I have administered a three node openshift v3 cluster in my lab environment.</p> <p>The summary of this expirence is that openshift is a great idea, but not ready for production. As an administrator you will find this a frustrating, difficult experience.</p> The minssf trap Wed, 23 Nov 2016 00:00:00 +0000 Unknown https://fy.blackhats.net.au/blog/2016-11-23-the-minssf-trap/ https://fy.blackhats.net.au/blog/2016-11-23-the-minssf-trap/ <h1 id="the-minssf-trap">The minssf trap</h1> <p>In directory server, we often use the concept of a &quot;minssf&quot;, or the &quot;minimum security strength factor&quot;. This is derived from cyrus sasl. However, there are some issues that will catch you out!</p> The mysterious crashing of my laptop Wed, 21 Sep 2016 00:00:00 +0000 Unknown https://fy.blackhats.net.au/blog/2016-09-21-the-mysterious-crashing-of-my-laptop/ https://fy.blackhats.net.au/blog/2016-09-21-the-mysterious-crashing-of-my-laptop/ <h1 id="the-mysterious-crashing-of-my-laptop">The mysterious crashing of my laptop</h1> <p>Recently I have grown unhappy with Fedora. The updates to the i915 graphics driver have caused my laptop to kernel panic just connecting and removing external displays: unacceptable to someone who moves their laptop around as much as I do.</p> What's new in 389 Directory Server 1.3.5 (unofficial) Wed, 21 Sep 2016 00:00:00 +0000 Unknown https://fy.blackhats.net.au/blog/2016-09-21-what-s-new-in-directory-server-1-3-5-unofficial/ https://fy.blackhats.net.au/blog/2016-09-21-what-s-new-in-directory-server-1-3-5-unofficial/ <h1 id="what-s-new-in-389-directory-server-1-3-5-unofficial">What's new in 389 Directory Server 1.3.5 (unofficial)</h1> <p>As a member of the 389 Directory Server (389DS) core team, I am always excited about our new releases. We have some really great features in 1.3.5. However, our changelogs are always large so I want to just touch on a few of my favourites.</p> <p>389 Directory Server is an LDAPv3 compliant server, used around the world for Identity Management, Authentication, Authorisation and much more. It is the foundation of the FreeIPA project's server. As a result, it's not something we often think about or even get excited for: but every day many of us rely on 389 Directory Server to be correct, secure and fast behind the scenes.</p> Block Chain for Identity Management Mon, 18 Jul 2016 00:00:00 +0000 Unknown https://fy.blackhats.net.au/blog/2016-07-18-block-chain-for-identity-management/ https://fy.blackhats.net.au/blog/2016-07-18-block-chain-for-identity-management/ <h1 id="block-chain-for-identity-management">Block Chain for Identity Management</h1> <p>On Sunday evening I was posed with a question and view of someone interested in Block Chain.</p> <p><em>&quot;What do you think of block chain for authentication&quot;</em></p> <p>A very heated debate ensued. I want to discuss this topic at length.</p> <h1 id="when-you-roll-x">When you roll X ...</h1> <p>We've heard this. &quot;When writing cryptography, don't. If you have to, hire a cryptographer&quot;.</p> <p><em>This statement is true for Authentication and Authorisation systems</em></p> <p>&quot;When writing Authentication and Authorisation systems, don't. If you have to, hire an Authentication expert&quot;.</p> <p>Guess what. Authentication and Authorisation are <em>hard</em>. Very hard. This is not something for the kids to play with; this is a security critical, business critical, sensitive, scrutinised and highly conservative area of technology.</p> Can I cycle through operators in C? Sat, 16 Jul 2016 00:00:00 +0000 Unknown https://fy.blackhats.net.au/blog/2016-07-16-can-i-cycle-through-operators-in-c/ https://fy.blackhats.net.au/blog/2016-07-16-can-i-cycle-through-operators-in-c/ <h1 id="can-i-cycle-through-operators-in-c">Can I cycle through operators in C?</h1> <p>A friend of mine who is learning to program asked me the following:</p> <p><em>&quot;&quot;&quot; How do i cycle through operators in c? If I want to try every version of + -</em> / on an equation, so printf(&quot;1 %operator 1&quot;, oneOfTheOperators); Like I'd stick them in an array and iterate over the array incrementing on each iteration, but do you define an operator as? They aren't ints, floats etc &quot;&quot;&quot;*</p> <p>He's certainly on the way to the correct answer already. There are three key barriers to an answer.</p> tracking down insane memory leaks Wed, 13 Jul 2016 00:00:00 +0000 Unknown https://fy.blackhats.net.au/blog/2016-07-13-tracking-down-insane-memory-leaks/ https://fy.blackhats.net.au/blog/2016-07-13-tracking-down-insane-memory-leaks/ <h1 id="tracking-down-insane-memory-leaks">tracking down insane memory leaks</h1> <p>One of the best parts of AddressSanitizer is the built in leak sanitiser. However, sometimes it's not as clear as you might wish!</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>I0&gt; /opt/dirsrv/bin/pwdhash hello </span><span>{SSHA}s16epVgkKenDHQqG8hrCGhmzqkgx0H1984ttYg== </span><span> </span><span>================================================================= </span><span>==388==ERROR: LeakSanitizer: detected memory leaks </span><span> </span><span>Direct leak of 72 byte(s) in 1 object(s) allocated from: </span><span> #0 0x7f5f5f94dfd0 in calloc (/lib64/libasan.so.3+0xc6fd0) </span><span> #1 0x7f5f5d7f72ae (/lib64/libnss3.so+0x752ae) </span><span> </span><span>SUMMARY: AddressSanitizer: 72 byte(s) leaked in 1 allocation(s). </span></code></pre> <p>&quot;Where is /lib64/libnss3.so+0x752ae&quot; and what can I do with it? I have debuginfo and devel info installed, but I can't seem to see what line that's at.</p> NSS and OpenSSL Command Reference Sun, 10 Jul 2016 00:00:00 +0000 Unknown https://fy.blackhats.net.au/pages/nss-and-openssl-command-reference/ https://fy.blackhats.net.au/pages/nss-and-openssl-command-reference/ <h1 id="nss-and-openssl-command-reference">NSS and OpenSSL Command Reference</h1> <p>I am tired of the lack of documentation for how to actually use OpenSSL and NSS to achieve things. Be it missing small important options like &quot;subjectAltNames&quot; in nss commands or OpenSSL's cryptic settings. Here is my complete list of everything you would ever want to do with OpenSSL and NSS.</p> <p>References:</p> <ul> <li><a href="http://www.mozilla.org/projects/security/pki/nss/tools/certutil.html">certutil mozilla</a>.</li> <li><a href="https://developer.mozilla.org/en-US/docs/NSS_reference/NSS_tools_:_certutil">nss tools</a>.</li> <li><a href="https://www.openssl.org/docs/apps/openssl.html">openssl</a>.</li> </ul> <h2 id="nss-specific">NSS specific</h2> <h3 id="db-creation-and-basic-listing">DB creation and basic listing</h3> <p>NSS does not operate on PEM or DER files like OpenSSL - it interacts with a certificate database.</p> <p>The older format database contains 3 files: key3.db, cert8.db, and secmod.db</p> <p>The newer sqlite based format contains 3 files: key4.db, cert9.db, and pkcs11.txt</p> <p>Create a new certificate database if one doesn't exist. You should see the files listed above. :</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>certutil -N -d . </span></code></pre> <p>List all certificates in a database :</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>certutil -L -d . </span></code></pre> <p>List all private keys in a database :</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>certutil -K -d . [-f pwdfile.txt] </span></code></pre> <p>I have created a password file, which consists of random data on one line in a plain text file. Something like below would suffice. Alternately you can enter a password when prompted by the certutil commands. If you wish to use this for directory server start up, you need to use pin.txt which lists the &quot;token name&quot; and it's corresponding pin. :</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>echo &quot;Password&quot; &gt; pwdfile.txt </span><span>echo &quot;internal:Password&quot; &gt; pin.txt </span></code></pre> <h3 id="importing-certificates-to-nss">Importing certificates to NSS</h3> <p>Import a server certificate into the database with no certificate authority trust flags. :</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>certutil -A -n &quot;Server-cert&quot; -t &quot;,,&quot; -i nss.dev.example.com.crt -d . </span></code></pre> <p>Import an openSSL generated key and certificate into an NSS database. This needs to be formatted through a pkcs12 bundle.</p> <p>You can NOT include the CA certificate in these bundles, they must be imported seperately to NSS. :</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>openssl pkcs12 -export -in server.crt -inkey server.key -out server.p12 -name Test-Server-Cert </span><span>pk12util -i server.p12 -d . -k pwdfile.txt </span></code></pre> <h3 id="importing-a-ca-certificate">Importing a CA certificate</h3> <p>Import the CA public certificate into the database. Note the [-t &quot;C,,&quot;]{.title-ref} flag which specifies this is trusted as a CA certificate. The nickname has no function other than to help you identify the certificate. :</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>certutil -A -n &quot;CAcert&quot; -t &quot;C,,&quot; -i /etc/pki/CA/nss/ca.crt -d . </span></code></pre> <h3 id="exporting-certificates">Exporting certificates</h3> <p>Export a secret key and certificate from an NSS database for use with OpenSSL. This must pass through a pkcs12 file, in reverse to the import process above. Note that file.pem contains all of the CA cert, cert and private key. :</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>pk12util -o server-export.p12 -d . -k pwdfile.txt -n Test-Server-Cert </span><span>openssl pkcs12 -in server-export.p12 -out file.pem -nodes </span></code></pre> <p>You can extract just the private key with:</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>openssl pkcs12 -in server-export.p12 -out file.pem -nocerts -nodes </span></code></pre> <p>Or just the cert and CAcert with</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>openssl pkcs12 -in server-export.p12 -out file.pem -nokeys -nodes </span></code></pre> <h2 id="both-nss-and-openssl">Both NSS and OpenSSL</h2> <h3 id="self-signed-certificates">Self signed certificates</h3> <p>Create a self signed certificate.</p> <p>For NSS, note the -n, which creates a &quot;nickname&quot; (should be unique in this DB) and is how applications reference your certificate and key. Also note the -s line, and the CN options. Finally, note the first line has the option -g, which defines the number of bits in the created certificate. Note also the -Z for the hash algorithm. -v is &quot;valid months&quot; which we set to 1. The equivalent openssl command is below.</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>certutil -S -f pwdfile.txt -d . -t &quot;C,,&quot; -x -n &quot;Server-Cert&quot; -k ec -q nistp256 \ </span><span>-s &quot;CN=nss.dev.example.com,O=Testing,L=example,ST=Queensland,C=AU&quot; -v 1 </span><span> </span><span>openssl req -x509 -newkey ec:&lt;(openssl genpkey -genparam -algorithm ec -pkeyopt ec_paramgen_curve:P-256) </span><span>-keyout key.pem -out cert.pem -days 31 -nodes </span></code></pre> <p>If you choose to use RSA. 3072 bits is the equivalent in strength to 256 bit ecdsa.</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>certutil -S -f pwdfile.txt -d . -t &quot;C,,&quot; -x -n &quot;Server-Cert&quot; -g 3072 -Z SHA256 \ </span><span>-s &quot;CN=nss.dev.example.com,O=Testing,L=example,ST=Queensland,C=AU&quot; -v 1 </span><span> </span><span>openssl req -x509 -newkey rsa:3072 -keyout key.pem -out cert.pem -days 31 -nodes </span></code></pre> <h3 id="subjectaltnames">SubjectAltNames</h3> <p>To add subject alternative names, use a comma seperated list with the option -8 IE:</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>certutil -S -f pwdfile.txt -d . -t &quot;C,,&quot; -x -n &quot;Server-Cert&quot; -k ec -q nistp256 \ </span><span>-s &quot;CN=nss.dev.example.com,O=Testing,L=example,ST=Queensland,C=AU&quot; \ </span><span>-8 &quot;nss.dev.example.com,nss-alt.dev.example.com&quot; </span><span> </span><span>openssl req -x509 -newkey ec:&lt;(openssl genpkey -genparam -algorithm ec -pkeyopt ec_paramgen_curve:P-256) \ </span><span>-keyout key.pem -out cert.pem -days 31 -subj &quot;/C=AU/CN=server.example.com&quot; \ </span><span>-addext &quot;subjectAltName = DNS:server.example.com&quot; \ </span><span>-addext &quot;extendedKeyUsage = serverAuth&quot; -nodes </span></code></pre> <p>For OpenSSL there is an alternative method: First, you need to create an altnames.cnf and fill in the details as required.</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>[req] </span><span>req_extensions = v3_req </span><span>nsComment = &quot;Certificate&quot; </span><span>distinguished_name = req_distinguished_name </span><span> </span><span>[ req_distinguished_name ] </span><span> </span><span>countryName = Country Name (2 letter code) </span><span>countryName_default = AU </span><span>countryName_min = 2 </span><span>countryName_max = 2 </span><span> </span><span>stateOrProvinceName = State or Province Name (full name) </span><span>stateOrProvinceName_default = Queensland </span><span> </span><span>localityName = Locality Name (eg, city) </span><span>localityName_default = example/streetAddress=Level </span><span> </span><span>0.organizationName = Organization Name (eg, company) </span><span>0.organizationName_default = example </span><span> </span><span>organizationalUnitName = Organizational Unit Name (eg, section) </span><span>organizationalUnitName_default = TS </span><span> </span><span>commonName = Common Name (eg, your name or your server\&#39;s hostname) </span><span>commonName_max = 64 </span><span> </span><span>[ v3_req ] </span><span> </span><span># Extensions to add to a certificate request </span><span> </span><span>basicConstraints = CA:FALSE </span><span>keyUsage = nonRepudiation, digitalSignature, keyEncipherment </span><span>subjectAltName = @alt_names </span><span> </span><span>[alt_names] </span><span>DNS.1 = server1.yourdomain.tld </span><span>DNS.2 = mail.yourdomain.tld </span><span>DNS.3 = www.yourdomain.tld </span><span>DNS.4 = www.sub.yourdomain.tld </span><span>DNS.5 = mx.yourdomain.tld </span><span>DNS.6 = support.yourdomain.tld </span></code></pre> <p>Now you run a similar command to before with the altnames configuration added. :</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>openssl req -x509 -newkey ec:&lt;(openssl genpkey -genparam -algorithm ec -pkeyopt ec_paramgen_curve:P-256) </span><span>-keyout key.pem -out cert.pem -days -config altnames.cnf </span></code></pre> <h3 id="check-a-certificate-belongs-to-a-specific-key">Check a certificate belongs to a specific key</h3> <p>This checks that a key, signing request and cert belong together.</p> <p>In NSS when the certificate and key are in the same database, the linkage is shown when you display all keys: :</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span># certutil -d . -K </span><span>&lt; 0&gt; ec bb4db46fb8a5beb46f57641f8b1bf236bc139666 NSS Certificate DB:Server-Cert </span></code></pre> <p>With OpenSSL it's possible to verify this from requests and other parts.</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>openssl ec -in key.pem -pubout | openssl sha1 </span><span>openssl x509 -noout -in cert.pem -pubkey | openssl sha1 </span><span>openssl req -noout -in cert.pem -pubkey | openssl sha1 </span></code></pre> <p>For an RSA key and certificate. :</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>openssl rsa -noout -modulus -in client.key | openssl sha1 </span><span>openssl req -noout -modulus -in client.csr | openssl sha1 </span><span>openssl x509 -noout -modulus -in client.crt | openssl sha1 </span></code></pre> <h3 id="view-a-certificate">View a certificate</h3> <p>View the cert :</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>certutil -L -d . -n Test-Cert </span><span> </span><span>openssl x509 -noout -text -in client.crt </span></code></pre> <p>View the cert in ASCII PEM form (This can be redirected to a file for use with openssl) :</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>certutil -L -d . -n Test-Cert -a </span><span>certutil -L -d . -n Test-Cert -a &gt; cert.pem </span></code></pre> <h3 id="creating-a-csr">Creating a CSR</h3> <p>In a <em>seperate</em> database to your CA.</p> <p>Create a new certificate request. Again, remember -8 for subjectAltName. This request is for a TLS server with a 24 month certificate lifetime. :</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>certutil -d . -R -a -o nss.dev.example.com.csr -f pwdfile.txt -k ec -q nistp256 -v 24 \ </span><span>-s &quot;CN=nss.dev.example.com,O=Testing,L=example,ST=Queensland,C=AU&quot; </span></code></pre> <p>If you want to request for a TLS client that can authenticate to a server with x509. :</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>certutil -d . -R -a -o user.csr -f pwdfile.txt -k ec -q nistp256 -v 24 \ </span><span>--keyUsage digitalSignature,nonRepudiation,keyEncipherment,dataEncipherment --nsCertType sslClient --extKeyUsage clientAuth \ </span><span>-s &quot;CN=username,O=Testing,L=example,ST=Queensland,C=AU&quot; </span></code></pre> <p>Using openSSL create a server key, and make a CSR. Note prime256v1 is an alternate name for nistp256 :</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>openssl ecparam -genkey -name prime256v1 -noout -out key.pem </span><span>openssl req -key key.pem -out cert.csr -days 712 -config altnames.cnf -new </span></code></pre> <p>For RSA :</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>openssl genrsa -out key.pem 3072 </span><span>openssl req -key key.pem -out cert.csr -days 712 -config altnames.cnf -new </span></code></pre> <h3 id="self-signed-ca">Self signed CA</h3> <p>Create a self signed CA (In a different database from the one used by your application) :</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>certutil -S -n CAissuer -t &quot;C,C,C&quot; -x -f pwdfile.txt -d . -k ec -q nistp256 -v 24 \ </span><span>--keyUsage certSigning -2 --nsCertType sslCA \ </span><span>-s &quot;CN=ca.nss.dev.example.com,O=Testing,L=example,ST=Queensland,C=AU&quot; </span></code></pre> <p>Nss will ask you about the constraints on this certificate. Here is a sample output. Note the path length of 0 still allows this CA to issue certificates, but it cannot issue an intermediate CA.</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>Generating key. This may take a few moments... </span><span> </span><span> 0 - Digital Signature </span><span> 1 - Non-repudiation </span><span> 2 - Key encipherment </span><span> 3 - Data encipherment </span><span> 4 - Key agreement </span><span> 5 - Cert signing key </span><span> 6 - CRL signing key </span><span> Other to finish </span><span> &gt; 5 </span><span> 0 - Digital Signature </span><span> 1 - Non-repudiation </span><span> 2 - Key encipherment </span><span> 3 - Data encipherment </span><span> 4 - Key agreement </span><span> 5 - Cert signing key </span><span> 6 - CRL signing key </span><span> Other to finish </span><span> &gt; 9 </span><span>Is this a critical extension [y/N]? </span><span>n </span><span>Is this a CA certificate [y/N]? </span><span>y </span><span>Enter the path length constraint, enter to skip [&lt;0 for unlimited path]: &gt; 0 </span><span>Is this a critical extension [y/N]? </span><span>y </span><span> 0 - SSL Client </span><span> 1 - SSL Server </span><span> 2 - S/MIME </span><span> 3 - Object Signing </span><span> 4 - Reserved for future use </span><span> 5 - SSL CA </span><span> 6 - S/MIME CA </span><span> 7 - Object Signing CA </span><span> Other to finish </span><span> &gt; 5 </span><span> 0 - SSL Client </span><span> 1 - SSL Server </span><span> 2 - S/MIME </span><span> 3 - Object Signing </span><span> 4 - Reserved for future use </span><span> 5 - SSL CA </span><span> 6 - S/MIME CA </span><span> 7 - Object Signing CA </span><span> Other to finish </span><span> &gt; 9 </span><span>Is this a critical extension [y/N]? </span><span>n </span></code></pre> <p>OpenSSL is the same as a self signed cert. It's probably wise to add path length and other policies here, which are specified via -config :</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>openssl req -x509 -newkey rsa:2048 -keyout key.pem -out cert.pem -days X -config ca.cnf </span></code></pre> <h3 id="renewing-the-self-signed-ca">Renewing the self signed CA</h3> <p>This happens if your CA is about to or has expired. You need to reissue all your certs after this is done! Be sure to substitute your domain and certificate nicknames.</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>certutil -d . -R -k &quot;NSS Certificate DB:ca&quot; -s &quot;CN=ca.net.blackhats.net.au,O=Blackhats,L=Brisbane,ST=Queensland,C=AU&quot; -a -o renew.req -1 -2 -5 </span><span> </span><span>certutil -C -d . -c &quot;ca&quot; -a -i renew.req -t &quot;C,C,C&quot; -o cacert.crt -v 12 </span><span> </span><span>certutil -A -d . -n &quot;ca&quot; -a -i cacert.crt -t &quot;C,C,C&quot; </span></code></pre> <h2 id="signing-with-the-ca">Signing with the CA</h2> <p>Create a certificate in the same database, and sign it with the CAissuer certificate.</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>certutil -S -n Test-Cert -t &quot;,,&quot; -c CAissuer -f pwdfile.txt -d . \ </span><span>-s &quot;CN=test.nss.dev.example.com,O=Testing,L=example,ST=Queensland,C=AU&quot; </span></code></pre> <p>If from a CSR, review the CSR you have recieved.</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>/usr/lib[64]/nss/unsupported-tools/derdump -i /etc/httpd/alias/nss.dev.example.com.csr </span><span>openssl req -inform DER -text -in /etc/httpd/alias/nss.dev.example.com.csr ## if from nss </span><span>openssl req -inform PEM -text -in server.csr ## if from openssl </span></code></pre> <p>On the CA, sign the CSR.</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>certutil -C -d . -f pwdfile.txt -a -i /etc/httpd/alias/nss.dev.example.com.csr \ </span><span>-o /etc/httpd/alias/nss.dev.example.com.crt -c CAissuer </span></code></pre> <p>For openssl CSR, note the use of -a that allows an ASCII formatted PEM input, and will create and ASCII PEM certificate output.</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>certutil -C -d . -f pwdfile.txt -i server.csr -o server.crt -a -c CAissuer </span><span> </span><span>### Note, you may need a caserial file ... </span><span>openssl x509 -req -days 1024 -in client.csr -CA root.crt -CAkey root.key -out client.crt </span></code></pre> <h3 id="check-validity-of-a-certificate">Check validity of a certificate</h3> <p>Test the new cert for validity as an SSL server. This assumes the CA cert is in the DB. (Else you need openssl or to import it). The second example is validating a user certificate.</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>certutil -V -d . -n Test-Cert -u V </span><span> </span><span>certutil -V -d . -n usercert -u C </span><span> </span><span>openssl verify -verbose -CAfile ca.crt client.crt </span></code></pre> <h3 id="export-the-ca-certificate">Export the CA certificate</h3> <p>Export the CA public certificate :</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>certutil -L -d . -n CAissuer -r &gt; ca.crt </span></code></pre> <h2 id="nss-sqlite-db">NSS sqlite db</h2> <p>Finally, these commands all use the old DBM formatted NSS databases. To use the new &quot;shareable&quot; sqlite formatting, follow the steps found from <a href="https://blogs.oracle.com/meena/entry/what_s_new_in_nss">this blog post</a>.</p> <p>How to upgrade from cert8.db to cert9.db</p> <p>You can either use environment variables or use sql: prefix in database directory parameter of certutil:</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>$export NSS_DEFAULT_DB_TYPE=sql </span><span>$certutil -K -d /tmp/nss -X </span><span> </span><span> OR </span><span> </span><span>$certutil -K -d sql:/tmp/nss -X </span></code></pre> <p>When you upgrade these are the files you get</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>key3.db -&gt; key4.db </span><span>cert8.db -&gt; cert9.db </span><span>secmod.db -&gt; pkcs11.txt </span></code></pre> <p>The contents of the pkcs11.txt files are basically identical to the contents of the old secmod.db, just not in the old Berkeley DB format. If you run the command &quot;$modutil -dbdir DBDIR -rawlist&quot; on an older secmod.db file, you should get output similar to what you see in pkcs11.txt.</p> <p>What needs to be done in programs / C code</p> <p>Either add environment variable NSS_DEFAULT_DB_TYPE &quot;sql&quot;</p> <p>NSS_Initialize call in <a href="https://developer.mozilla.org/en/NSS_Initialize">https://developer.mozilla.org/en/NSS_Initialize</a> takes this &quot;configDir&quot; parameter as shown below.</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>NSS_Initialize(configDir, &quot;&quot;, &quot;&quot;, &quot;secmod.db&quot;, NSS_INIT_READONLY); </span></code></pre> <p>For cert9.db, change this first parameter to &quot;sql:&quot; + configDir (like &quot;sql:/tmp/nss/&quot;) i.e. prefix &quot;sql:&quot; in the directory name where these NSS Databases exist. This code will work with cert8.db as well if cert9.db is not present.</p> <p><a href="https://wiki.mozilla.org/NSS_Shared_DB">https://wiki.mozilla.org/NSS_Shared_DB</a></p> <h2 id="display-a-human-readable-certificate-from-an-ssl-socket">Display a human readable certificate from an SSL socket</h2> <p>Note: port 636 is LDAPS, but all SSL sockets are supported. For TLS only a limited set of protocols are supported. Add -starttls to the command. See man 1 s_client.</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>openssl s_client -connect ldap.example.com:636 </span><span> </span><span>[ant@ant-its-example-edu-au ~]$ echo -n | openssl s_client -connect ldap.example.com:636 | sed -ne &#39;/-BEGIN CERTIFICATE-/,/-END CERTIFICATE-/p&#39; | openssl x509 -noout -text </span><span> </span><span>depth=3 C = SE, O = AddTrust AB, OU = AddTrust External TTP Network, CN = AddTrust External CA Root </span><span>verify return:1 </span><span>depth=2 C = US, ST = UT, L = Salt Lake City, O = The USERTRUST Network, OU = http://www.usertrust.com, CN = UTN-USERFirst-Hardware </span><span>verify return:1 </span><span>depth=1 C = AU, O = AusCERT, OU = Certificate Services, CN = AusCERT Server CA </span><span>verify return:1 </span><span>depth=0 C = AU, postalCode = 5000, ST = Queensland, L = example, street = Level, street = Place, O =Example, OU = Technology Services, CN = ldap.example.com </span><span>verify return:1 </span><span>DONE </span><span>Certificate: </span><span> Data: </span><span> Version: 3 (0x2) </span><span> Serial Number: </span><span> Signature Algorithm: sha1WithRSAEncryption </span><span> Issuer: C=AU, O=AusCERT, OU=Certificate Services, CN=AusCERT Server CA </span><span> Validity </span><span> Not Before: XX </span><span> Not After : XX </span><span> Subject: C=AU/postalCode=4000, ST=Queensland, L=example/street=Level /street=Place, O=Example, OU=Technology Services, CN=ldap.example.com </span><span> Subject Public Key Info: </span><span>&lt;snip&gt; </span><span> X509v3 Subject Alternative Name: </span><span> DNS:ldap.example.com </span><span>&lt;snip&gt; </span></code></pre> <p>You can use this to display a CA chain if you can't get it from other locations.</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>openssl s_client -connect ldap.example.com:636 -showcerts </span></code></pre> LDAP Guide Part 4: Schema and Objects Fri, 08 Jul 2016 00:00:00 +0000 Unknown https://fy.blackhats.net.au/pages/ldap-guide-part-4-schema-and-objects/ https://fy.blackhats.net.au/pages/ldap-guide-part-4-schema-and-objects/ <h1 id="ldap-guide-part-4-schema-and-objects">LDAP Guide Part 4: Schema and Objects</h1> <p>So far we have seen that LDAP is a tree based database, that allows complex filters over objects attribute value pairs.</p> <p>Unlike a no-sql or schemaless database, LDAP has a schema for it's objects, making it stricter than json or other similar-looking representations. This schema is based on objectClasses, similar to object-oriented programming.</p> <h2 id="searching-the-schema">Searching the schema</h2> <p>Sadly, schema is a bit difficult to parse due to it's representation as a single object. You can show all the objectClasses and attributeTypes definitions with the following search.</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>ldapsearch -H ldap://exampleldap.blackhats.net.au:3389 -x -b &#39;cn=schema&#39; &#39;(objectClass=*)&#39; + </span></code></pre> <p><em>note</em>: We have a tool in development that makes searching for these details easier, but we haven't released it yet.</p> <p>You'll notice two important types here. The first is an attributeType:</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>attributeTypes: ( 2.5.4.4 NAME ( &#39;sn&#39; &#39;surName&#39; ) SUP name </span><span> EQUALITY caseIgnoreMatch SUBSTR caseIgnoreSubstringsMatch </span><span> SYNTAX 1.3.6.1.4.1.1466.115.121.1.15 X-ORIGIN &#39;RFC 4519&#39; </span><span> X-DEPRECATED &#39;surName&#39; ) </span></code></pre> <p>This is the definition of how an attribute is named and represented. We can see this attribute can be named &quot;sn&quot; or &quot;surName&quot; (but surName is deprecated), it uses a case-insensitive match for checks (ie Brown and brown are the same), and the syntax oid defines the data this can hold: in this case a utf8 string.</p> <p>For the most part you won't need to play with attributeTypes unless you have an odd data edge case you are trying to represent.</p> <p>The second important type is the objectClass definition:</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>objectClasses: ( 2.5.6.6 NAME &#39;person&#39; SUP top STRUCTURAL MUST ( sn $ cn ) </span><span> MAY ( userPassword $ telephoneNumber $ seeAlso $ description ) </span><span> X-ORIGIN &#39;RFC 4519&#39; ) </span></code></pre> <p>This defines an objectClass called &quot;person&quot;. It's parent is the &quot;top&quot; objectClass, and for this to exist on an object the object MUST have sn and cn attributes present. You may optionally include the MAY attributes on the object also.</p> <h2 id="example">Example</h2> <p>So using our person objectclass we can create a simple object:</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>cn=user,dc=... </span><span>objectClass: top </span><span>objectClass: person </span><span>cn: user </span><span>sn: user </span></code></pre> <p>Let's go through a few things. First, note that the rdn (cn=user), is a valid attribute on the object (cn: user). If we omitted this or changed it, it would not be valid. This for example is invalid:</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>cn=user,dc=... </span><span>objectClass: top </span><span>objectClass: person </span><span>cn: somethingelse </span><span>sn: user </span><span>description: invalid! </span></code></pre> <p>If we don't satisfy the schema's &quot;MUST&quot; requirements, the object is also invalid:</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>cn=user,dc=... </span><span>objectClass: top </span><span>objectClass: person </span><span>sn: user </span><span>description: invalid, missing cn! </span></code></pre> <p>It IS valid to add any of the MAY types to an object of course, but they can also be absent (as per the examples above):</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>cn=user,dc=... </span><span>objectClass: top </span><span>objectClass: person </span><span>cn: user </span><span>sn: user </span><span>telephoneNumber: 0118999.... </span><span>description: valid with may attrs. </span></code></pre> <h2 id="complex-objects">Complex objects</h2> <p>You are not limited to a single objectClass per object either. You can list multiple objectClasses:</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>objectClass: account </span><span>objectClass: inetOrgPerson </span><span>objectClass: inetUser </span><span>objectClass: ldapPublicKey </span><span>objectClass: ntUser </span><span>objectClass: organizationalPerson </span><span>objectClass: person </span><span>objectClass: posixaccount </span><span>objectClass: top </span></code></pre> <p>Provided that all the MUST requirements of all objectClasses are satisfied, this is valid.</p> <p>If an attribute exists in both a MUST and a MAY of an objectClass, then the stricter requirement is enforced, IE MUST. Here, both objectClasses define cn, but ldapsubentry defines it as &quot;MAY&quot; and person as &quot;MUST&quot;. Therfore, on an object that contained both of these, CN is a must attribute.</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>objectClasses: ( 2.16.840.1.113719.2.142.6.1.1 NAME &#39;ldapSubEntry&#39; DESC &#39;LDAP </span><span> Subentry class, version 1&#39; SUP top STRUCTURAL MAY cn X-ORIGIN &#39;LDAP Subentry </span><span> Internet Draft&#39; ) </span><span>objectClasses: ( 2.5.6.6 NAME &#39;person&#39; SUP top STRUCTURAL MUST ( sn $ cn ) </span><span> MAY ( userPassword $ telephoneNumber $ seeAlso $ description ) </span><span> X-ORIGIN &#39;RFC 4519&#39; ) </span></code></pre> <h2 id="building-your-own-objects">Building your own objects</h2> <p>Knowing this now you can use this to create your own objects. There are some common attributes that generally need to be satisfied to allow user objects to resolve on unix systems. You'll probably need:</p> <ul> <li>uid</li> <li>displayName</li> <li>loginShell</li> <li>homeDirectory</li> <li>uidNumber</li> <li>gidNumber</li> <li>memberOf</li> </ul> <p>For a group to resolve you need:</p> <ul> <li>gidNumber</li> <li>member</li> </ul> <p><em>NOTE</em>: This is assuming rfc2307bis behaviour for your client. In sssd this is &quot;ldap_schema = rfc2307bis&quot;, in the domain provider. For other clients you may need to alter other parameters. This is the most efficent way to resolve groups and users on unix, so strongly consider it.</p> <p>Knowing you need these values, you can search the schema to create objectClass definitions to match. Try this out:</p> <h2 id="answers">Answers</h2> <p>For the group, this is pretty easy. You should have:</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>objectClass: top </span><span>objectClass: posixGroup </span><span>objectClass: groupOfNames </span></code></pre> <p>The user is a bit tricker. You should have something similar to:</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>objectClass: top </span><span>objectClass: account </span><span>objectClass: person </span><span>objectClass: posixAccount </span></code></pre> <p>Remember, there is more than one way to put these objects together to have valid attributes that you require - just try to make sure you pick classes that don't have excess attributes. A bad choice for example is:</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>objectClass: top </span><span>objectClass: nsValueItem </span></code></pre> <p>nsValueItem gives you a &quot;MUST&quot; cn, but it gives &quot;MAY&quot; of many other attributes you will never use or need. So account or person are better choices. Generally the clue is in the objectClass name.</p> <p>If you have your own LDAP server you can try creating objects now with these classes.</p> LDAP Guide Part 3: Filters Wed, 06 Jul 2016 00:00:00 +0000 Unknown https://fy.blackhats.net.au/pages/ldap-guide-part-3-filters/ https://fy.blackhats.net.au/pages/ldap-guide-part-3-filters/ <h1 id="ldap-guide-part-3-filters">LDAP Guide Part 3: Filters</h1> <p>In part 2 we discussed how to use searchbases to control what objects were returned from a search by their organisation in the LDAP Tree. We also touched on a simple filter to limit the result by a single search item.</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>&#39;(cn=HR Managers)&#39; </span></code></pre> <p>If we change this to a different part of the tree, we'll get back too many entries:</p> <p><em>REMEMBER</em>: All examples in this page work and can be &quot;copy-pasted&quot; so you can try these searches for yourself!</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>ldapsearch -H ldap://exampleldap.blackhats.net.au:3389 -x -s sub -b &#39;ou=People,dc=example,dc=com&#39; </span></code></pre> <p>If this database has thousands of users, we wouldn't be able to scale or handle this. We need to be able to use filters to effectively search for objects.</p> <h2 id="simple-filters">Simple Filters</h2> <p>As mentioned before filters are of the format:</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>(attribute operator value) </span><span>(condition (...) ) </span></code></pre> <p>These filters are rooted in <a href="https://en.wikipedia.org/wiki/Set_(mathematics)">set mathematics</a> which may be good as an additional reference.</p> <p>Filters apply to objects attribute values - not the DN. Remember though, the RDN <em>must</em> be an attribute of the object, so you can filter on this. It's a good idea to look at an object to understand what you could filter on:</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span># test0002, People, example.com </span><span>dn: uid=test0002,ou=People,dc=example,dc=com </span><span>objectClass: top </span><span>objectClass: person </span><span>objectClass: organizationalPerson </span><span>objectClass: inetOrgPerson </span><span>objectClass: posixAccount </span><span>objectClass: shadowAccount </span><span>cn: guest0002 </span><span>sn: guest0002 </span><span>uid: guest0002 </span><span>uid: test0002 </span><span>givenName: givenname0002 </span><span>description: description0002 </span><span>mail: uid0002 </span><span>uidNumber: 2 </span><span>gidNumber: 2 </span><span>shadowMin: 0 </span><span>shadowMax: 99999 </span><span>shadowInactive: 30 </span><span>shadowWarning: 7 </span><span>homeDirectory: /home/uid0002 </span><span>shadowLastChange: 17427 </span></code></pre> <h2 id="equality-filters">Equality Filters</h2> <p>An equality filter requests all objects where attribute is equal to value. IE:</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>&#39;(uid=test0009)&#39; </span><span>ldapsearch -H ldap://exampleldap.blackhats.net.au:3389 -x -s sub -b &#39;ou=People,dc=example,dc=com&#39; &#39;(uid=test0009)&#39; </span></code></pre> <h2 id="presence-filter">Presence Filter</h2> <p>A presence filter requests all objects where the attribute is present and has a valid value, but we do not care what the value is.</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>&#39;(uid=*)&#39; </span><span>ldapsearch -H ldap://exampleldap.blackhats.net.au:3389 -x -s sub -b &#39;ou=People,dc=example,dc=com&#39; &#39;(uid=*)&#39; </span></code></pre> <h2 id="range-filters">Range Filters</h2> <p>A range filter requests all objects whose attribute values are greater than or less than a value.</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>&#39;(uid&gt;=test0005)&#39; </span><span>ldapsearch -H ldap://exampleldap.blackhats.net.au:3389 -x -s sub -b &#39;ou=People,dc=example,dc=com&#39; &#39;(uid&gt;=test0005)&#39; </span><span> </span><span>&#39;(uid&lt;=test0005)&#39; </span><span>ldapsearch -H ldap://exampleldap.blackhats.net.au:3389 -x -s sub -b &#39;ou=People,dc=example,dc=com&#39; &#39;(uid&lt;=test0005)&#39; </span></code></pre> <h2 id="substring-filters">Substring filters</h2> <p>This requests a partial match of an attribute value on the object. You can use the '*' operator multiple times.</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>&#39;(uid=*005)&#39; </span><span>ldapsearch -H ldap://exampleldap.blackhats.net.au:3389 -x -s sub -b &#39;ou=People,dc=example,dc=com&#39; &#39;(uid=*005)&#39; </span><span>&#39;(uid=*st000*)&#39; </span><span>ldapsearch -H ldap://exampleldap.blackhats.net.au:3389 -x -s sub -b &#39;ou=People,dc=example,dc=com&#39; &#39;(uid=*st000*)&#39; </span></code></pre> <p>NOTE: You should always have at least 3 characters in your substring filter, else indexes may not operate efficently. IE this filter may not work efficently:</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>&#39;(uid=*05)&#39; </span></code></pre> <h2 id="and-filters">AND filters</h2> <p>Using the filters above we can begin to construct more complex queries. AND requires that for an object to match, all child filter elements must match. This is the &quot;intersection&quot; operation.</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>&#39;(&amp;(uid=test0006)(uid=guest0006))&#39; </span><span>ldapsearch -H ldap://exampleldap.blackhats.net.au:3389 -x -s sub -b &#39;ou=People,dc=example,dc=com&#39; &#39;(&amp;(uid=test0006)(uid=guest0006))&#39; </span></code></pre> <p>Because the object has both uid=test0006 and uid=guest0006, this returns the object uid=test0006,ou=People,dc=example,dc=com. However, if we changed this condition:</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>&#39;(&amp;(uid=test0006)(uid=guest0007))&#39; </span><span>ldapsearch -H ldap://exampleldap.blackhats.net.au:3389 -x -s sub -b &#39;ou=People,dc=example,dc=com&#39; &#39;(&amp;(uid=test0006)(uid=guest0007))&#39; </span></code></pre> <p>No objects match both predicates, so we have an empty result set.</p> <h2 id="or-filters">OR filters</h2> <p>OR filters will return the aggregate of all child filters. This is the union operation. Provided an object satisfies one condition of the OR, it will be part of the returned set.</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>&#39;(|(uid=test0006)(uid=guest0007))&#39; </span><span>ldapsearch -H ldap://exampleldap.blackhats.net.au:3389 -x -s sub -b &#39;ou=People,dc=example,dc=com&#39; &#39;(|(uid=test0006)(uid=guest0007))&#39; </span></code></pre> <p>If an object is matched twice in the OR filter, we only return it once.</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>&#39;(|(uid=test0008)(uid=guest0008))&#39; </span><span>ldapsearch -H ldap://exampleldap.blackhats.net.au:3389 -x -s sub -b &#39;ou=People,dc=example,dc=com&#39; &#39;(|(uid=test0008)(uid=guest0008))&#39; </span></code></pre> <h2 id="not-filters">NOT filters</h2> <p>A NOT filter acts to invert the result of the inner set. NOT is the equivalent of a negating AND. For example:</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>&#39;(!(uid=test0010))&#39; </span><span>ldapsearch -H ldap://exampleldap.blackhats.net.au:3389 -x -s sub -b &#39;ou=People,dc=example,dc=com&#39; &#39;(!(uid=test0010))&#39; </span></code></pre> <p>You can't list multiple parameters to a not condition however:</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>&#39;(!(uid=test0010)(uid=test0009))&#39; </span><span>ldapsearch -H ldap://exampleldap.blackhats.net.au:3389 -x -s sub -b &#39;ou=People,dc=example,dc=com&#39; &#39;(!(uid=test0010)(uid=test0009))&#39; </span><span>... </span><span>ldap_search_ext: Bad search filter (-7) </span></code></pre> <p>To combine NOT's you need to use this in conjunction with AND and OR.</p> <h2 id="complex-filters">Complex filters</h2> <p>Because AND OR and NOT are filters in their own right, you can nest these to produce more complex directed queries.</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>&#39;(&amp;(objectClass=person)(objectClass=posixAccount)(|(uid=test000*))(!(uid=test0001)))&#39; </span><span>ldapsearch -H ldap://exampleldap.blackhats.net.au:3389 -x -s sub -b &#39;ou=People,dc=example,dc=com&#39; &#39;(&amp;(objectClass=person)(objectClass=posixAccount)(|(uid=test000*))(!(uid=test0001)))&#39; uid </span></code></pre> <p>I find it useful to break this down to see what is happening</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>(&amp; </span><span> (objectClass=person) </span><span> (objectClass=posixAccount) </span><span> (| </span><span> (uid=test000*) </span><span> ) </span><span> (!(uid=test0001)) </span><span>) </span></code></pre> <p>This query expresses &quot;All person whose name starts with test000* and not test0001&quot;. Once broken down over multiple lines like this it's easy to see which filters belong to which logic components, and how they will interact.</p> <h2 id="conclusion">Conclusion</h2> <p>While search bases can help you to direct a query, filters are how searches are efficently expressed over databases of millions of objects. Being able to use them effectively will help your client applications be much faster.</p> LDAP Guide Part 2: Searching Tue, 05 Jul 2016 00:00:00 +0000 Unknown https://fy.blackhats.net.au/pages/the-ldap-guide-part-2-searching/ https://fy.blackhats.net.au/pages/the-ldap-guide-part-2-searching/ <h1 id="ldap-guide-part-2-searching">LDAP Guide Part 2: Searching</h1> <p>In the first part, we discussed how and LDAP tree is laid out, and why it's called a &quot;tree&quot;.</p> <p>In this part, we will discuss the most important and fundamental component of ldap: Searching.</p> <p>A note is that <em>all</em> examples and commands in this document <em>work</em>. I have established an internet facing ldap server with which you can query to test out searches. This will work on any RPM based system with openldap-clients installed, or OSX.</p> <p>To test connectivity you should see the following:</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>I0&gt; ldapsearch -H ldap://exampleldap.blackhats.net.au:3389 -b &#39;&#39; -s base -x -LLL vendorVersion </span><span>dn: </span><span>vendorVersion: 389-Directory/1.3.4.0 B2016.175.1716 </span></code></pre> <p>If you see any other errors, you have some issue with your network or environment.</p> <h2 id="introduction">Introduction</h2> <p>Remember that we have a tree of objects, organised by their RDN, the Relative Distinguished Name.</p> <p>An LDAP object looks like this:</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>dn: ou=Groups,dc=example,dc=com </span><span>objectClass: top </span><span>objectClass: organizationalunit </span><span>ou: Groups </span></code></pre> <p>We see the DN, which is built from the RDN components. We have a number of objectClasses that defined the structure and attributes of the object. Finally, we have the attribute &quot;ou&quot;, which in this case happens to be our RDN.</p> <p>A more &quot;complete&quot; object is this example:</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>dn: cn=Accounting Managers,ou=Groups,dc=example,dc=com </span><span>objectClass: top </span><span>objectClass: groupOfUniqueNames </span><span>cn: Accounting Managers </span><span>ou: groups </span><span>description: People who can manage accounting entries </span><span>uniqueMember: cn=Directory Manager </span></code></pre> <p>Again we can see the DN, the objectclasses, and the cn, which is our RDN. We also have a number of other attributes, such as ou, description, uniqueMember. This are <em>not</em> part of the RDN, but they are still parts of the object.</p> <h2 id="basic-searching">Basic searching</h2> <p>Because LDAP is a tree, we must define a basedn: The &quot;root&quot; or &quot;anchor&quot; point in the tree we want to search beneath. To show what basedns are avaliable, we can query the special '' or blank rootDSE (Directory Server Entry).</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>ldapsearch -H ldap://exampleldap.blackhats.net.au:3389 -b &#39;&#39; -s base -x namingContexts </span><span>... </span><span>namingContexts: dc=example,dc=com </span></code></pre> <p>We can now use this in our search command: Note the -b argument. This is the search basedn.</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>ldapsearch -H ldap://exampleldap.blackhats.net.au:3389 -x -b &#39;dc=example,dc=com&#39; </span></code></pre> <p>You should see a lot of data on your screen from that last command! We just showed every object in the tree. Here is the layout of the data in the exampleldap server to help you understand that output.</p> <p><img src="/_static/search-1.svg" alt="image" />{width=&quot;850px&quot;}</p> <h2 id="using-a-different-basedn">Using a different basedn</h2> <p>We are not just limited to using the basedn &quot;dc=example,dc=com&quot;. This returns a lot of data, so sometimes we might want to focus our search.</p> <p>By default LDAP is performing what is called a <em>subtree</em> search. A subtree search means &quot;include all entries including the basedn in my search&quot;.</p> <p>Lets say we wanted to see just the entries highlighted in blue.</p> <p><img src="/_static/search-2.svg" alt="image" />{width=&quot;850px&quot;}</p> <p>The solution is to <em>change</em> the basedn of our search.</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>ldapsearch -H ldap://exampleldap.blackhats.net.au:3389 -x -b &#39;ou=Groups,dc=example,dc=com&#39; </span></code></pre> <p>Now you should see that we only see the results highlighted in blue.</p> <p>You can try this for other parts of the directory too.</p> <h2 id="limiting-the-scope-of-the-search">Limiting the scope of the search</h2> <p>As previously mentioned, we are by default performing a subtree search for ldap. But perhaps we only wanted to focus our search further.</p> <p>This is controlled by the '-s' parameter to the ldapsearch command.</p> <p>In this case, we want only the nodes again, in blue. This time we want only the child entries of ou=Groups, but <em>not</em> ou=Groups itself.</p> <p><img src="/_static/search-3.svg" alt="image" />{width=&quot;850px&quot;}</p> <p>Now we need to limit not the basedn of the search, but the <em>scope</em>. The ldap search scope says which entries we should use. We have already discussed subtree. In this case we want to use the scope called <em>onelevel</em>. This means &quot;search entries that are direct children of the basedn only&quot;.</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>ldapsearch -H ldap://exampleldap.blackhats.net.au:3389 -x -s onelevel -b &#39;ou=Groups,dc=example,dc=com&#39; </span></code></pre> <p>From the result, you can see, we only see the entries again in blue.</p> <p>A key point of onelevel is the direct children only are searched. So were we to move the basedn back up to dc=example,dc=com, and perform a onelevel search, we will only see the following.</p> <p><img src="/_static/search-4.svg" alt="image" />{width=&quot;850px&quot;}</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>ldapsearch -H ldap://exampleldap.blackhats.net.au:3389 -x -s onelevel -b &#39;dc=example,dc=com&#39; </span></code></pre> <p>In addition to subtree and onelevel we have one more search scope. The final scope is named 'base'. This search scope returns <em>only</em> the basedn of the search.</p> <p>So if we were to want to retrieve a single entry by DN, this is how we would achieve that.</p> <p><img src="/_static/search-5.svg" alt="image" />{width=&quot;850px&quot;}</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>ldapsearch -H ldap://exampleldap.blackhats.net.au:3389 -x -s base -b &#39;ou=Groups,dc=example,dc=com&#39; </span></code></pre> <h2 id="filtering-a-set-of-objects">Filtering a set of objects</h2> <p>The most important part of a search is likely the filter. This defines a query where the objects returned must match the filter conditions.</p> <p>A filter applies to every attribute of every object within the search scope. IE It does not just apply to the RDN of the object.</p> <p>Filters take the form:</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>(attribute=value) </span></code></pre> <p>Filters can be nested also with special conditions. The condition applies to all filters that follow within the same level of brackets.</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>(condition (attribute=value)(attribute=value)) </span></code></pre> <p>By default, ldapsearch provides the filter</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>(objectClass=*) </span></code></pre> <p>* is a special value, representing &quot;any possible value&quot;. Because all objects must have an objectClass, this filter is the equivalent to saying &quot;all objects&quot;.</p> <p>You can see this doesn't change the output when we run these two commands:</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>ldapsearch -H ldap://exampleldap.blackhats.net.au:3389 -x -s sub -b &#39;ou=Groups,dc=example,dc=com&#39; &#39;(objectClass=*)&#39; </span><span>ldapsearch -H ldap://exampleldap.blackhats.net.au:3389 -x -s sub -b &#39;ou=Groups,dc=example,dc=com&#39; </span></code></pre> <p>If we were to want to retrieve <em>only</em> the HR Managers group, but we didn't know it's RDN / DN. Because we know it has the attribute &quot;cn=HR Managers&quot;, we can construct a filter that will retrieve &quot;any object where cn exactly matches the value HR Managers.</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>ldapsearch -H ldap://exampleldap.blackhats.net.au:3389 -x -s sub -b &#39;ou=Groups,dc=example,dc=com&#39; &#39;(cn=HR Managers)&#39; </span></code></pre> <p>Say that you did not know that the HR Managers group was in ou=Groups. The following would also be valid:</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>ldapsearch -H ldap://exampleldap.blackhats.net.au:3389 -x -s sub -b &#39;dc=example,dc=com&#39; &#39;(cn=HR Managers)&#39; </span></code></pre> <p>Thus, you often see queries using the base namingContext of the directory, but applying filters to limit the objects returned.</p> <p>More complex filters than this exist, and will be part 3 of this guide.</p> <h2 id="attributes-limiting-what-we-get-back">Attributes: Limiting what we get back</h2> <p>When we are searching, we often do not want the entire object returned to us. We only need to see one important piece of data. For our HR Managers group, we want to know who the members are. Recall the object is:</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>dn: cn=HR Managers,ou=Groups,dc=example,dc=com </span><span>objectClass: top </span><span>objectClass: groupOfUniqueNames </span><span>cn: HR Managers </span><span>ou: groups </span><span>description: People who can manage HR entries </span><span>uniqueMember: cn=Directory Manager </span></code></pre> <p>We only want to know who is in the uniqueMember attribute: We do not care for the other values.</p> <p>The final part of an ldapsearch is control of what attributes are returned. Once the scope and filters have been applied, the set of objects returned will only display the attributes in the list.</p> <p>To do this, you put a space seperated list at the end of the ldap search command:</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>ldapsearch -H ldap://exampleldap.blackhats.net.au:3389 -x -s sub -b &#39;dc=example,dc=com&#39; &#39;(cn=HR Managers)&#39; uniqueMember </span></code></pre> <p>You can return multiple attributes if you wish:</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>ldapsearch -H ldap://exampleldap.blackhats.net.au:3389 -x -s sub -b &#39;dc=example,dc=com&#39; &#39;(cn=HR Managers)&#39; uniqueMember cn </span></code></pre> <h2 id="conclusion">Conclusion</h2> <p>Ldapsearches tend to be very foreign to application developers and engineers when they first encounter them. Unlike SQL they are not based on tables and selects, and often the data is more complex is disparate. However with these controls, being basedn, scope, filter and attributes, you can completely direct your search to return the exact data that you require for your application or query.</p> <p><a href="https://fy.blackhats.net.au/pages/the-ldap-guide-part-2-searching/ldap_guide_part_3_filters.html">PART 3, filters!</a></p> <h2 id="notes">Notes:</h2> <p>The ldap server for this example is setup as:</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>sudo yum install -y 389-ds-base </span></code></pre> <p>example-setup.inf</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>[General] </span><span>FullMachineName = localhost.localdomain </span><span>ServerRoot = /lib/dirsrv </span><span>SuiteSpotGroup = dirsrv </span><span>SuiteSpotUserID = dirsrv </span><span>StrictHostCheck = false </span><span>[slapd] </span><span>AddOrgEntries = Yes </span><span>AddSampleEntries = No </span><span>HashedRootDNPwd = </span><span>InstallLdifFile = suggest </span><span>RootDN = cn=Directory Manager </span><span>RootDNPwd = </span><span>ServerIdentifier = example </span><span>ServerPort = 3389 </span><span>Suffix = dc=example,dc=com </span><span>bak_dir = /var/lib/dirsrv/slapd-example/bak </span><span>bindir = /bin </span><span>cert_dir = /etc/dirsrv/slapd-example </span><span>config_dir = /etc/dirsrv/slapd-example </span><span>datadir = /share </span><span>db_dir = /var/lib/dirsrv/slapd-example/db </span><span>ds_bename = userRoot </span><span>inst_dir = /lib/dirsrv/slapd-example </span><span>ldif_dir = /var/lib/dirsrv/slapd-example/ldif </span><span>localstatedir = /var </span><span>lock_dir = /var/lock/dirsrv/slapd-example </span><span>log_dir = /var/log/dirsrv/slapd-example </span><span>naming_value = example </span><span>run_dir = /var/run/dirsrv </span><span>sbindir = /sbin </span><span>schema_dir = /etc/dirsrv/slapd-example/schema </span><span>sysconfdir = /etc </span><span>tmp_dir = /tmp </span><span> </span><span>setup-ds.pl --silent --file example-setup.inf </span><span>firewall-cmd --zone=internal --add-service=ldap </span><span>systemctl enable dirsrv@example </span><span>systemctl stop dirsrv@example </span><span>db2ldif -Z example -n userRoot </span><span>crontab -e # Put in refresh.sh to run every 4 hours. </span></code></pre> <p>refresh.sh</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>#!/bin/bash </span><span>systemctl stop dirsrv@example </span><span>ldif2db -Z example -n userRoot -i /var/lib/dirsrv/slapd-example/ldif/example-userRoot-2016_07_05_103810.ldif </span><span>systemctl start dirsrv@example </span></code></pre> LDAP Guide Part 1: Foundations Mon, 20 Jun 2016 00:00:00 +0000 Unknown https://fy.blackhats.net.au/pages/the-ldap-guide-part-1-foundations/ https://fy.blackhats.net.au/pages/the-ldap-guide-part-1-foundations/ <h1 id="ldap-guide-part-1-foundations">LDAP Guide Part 1: Foundations</h1> <p>To understand LDAP we must understand a number of concepts of datastructures: Specifically graphs.</p> <h2 id="undirected">Undirected</h2> <p>In computer science, a set of nodes, connected by some set of edges is called a graph. Here we can see a basic example of a graph.</p> <p><img src="/_static/graph-basic-1.svg" alt="image" /></p> <p>Viewing this graph, we can see that it has a number of properties. It has 4 nodes, and 4 edges. As this is undirected we can assume the link A to B is as valid as B to A.</p> <p>We also have a cycle: That is a loop between nodes. We can see this in B, C, D. If any edge between the set of B, D or B, C, or C, D were removed, this graph would no longer have cycles.</p> <h2 id="directed">Directed</h2> <p>A directed graph is where each edge not only defines a link between two nodes, but also the direction of the link. For example, we can see that A to B is a valid edge, but B to A is not. We would say that the node where the link is from is the outgoing node of the edge. Where the node recieves an egde, IE the arrow, is an incoming edge.</p> <p><img src="/_static/graph-basic-2.svg" alt="image" /></p> <p>In this graph, for a cycle to occur, we must have a set of nodes where not only the edges exist, but the direction allows a loop. Here, the cycle is B, C, D. Were the link between C and D reversed, we no longer have a cycle in our directed graph.</p> <p><img src="/_static/graph-basic-3.svg" alt="image" /></p> <h2 id="trees">Trees</h2> <p>A tree is a special case of the directed graph. The properties of a tree are that:</p> <ul> <li>Each node has 1 and only 1 incoming edge.</li> <li>The graph may have no cycles.</li> </ul> <p>An example of a tree is below. You can check and it maintains all the properties above. Note there is no limit to outbound edges, the only rule is maximum of one incoming.</p> <p><img src="/_static/graph-basic-4.svg" alt="image" /></p> <p>A property that you regularly see is that nodes are unique in a tree, IE A will not appear twice. This allows for <em>searching</em> of the tree.</p> <h2 id="more-on-nodes">More on nodes</h2> <p>So far our nodes have been a bit bland. We can do more with them though. Instead of just storing a single datum in them, we can instead store the datum as a key to lookup the node, and then have more complex data in the value of the node. For example, we can expand our tree to look like this:</p> <p><img src="/_static/graph-basic-5.svg" alt="image" /></p> <p>This is why having unique keys in our nodes is important. It allows us to search the tree for that node, and to retrieve the data stored within.</p> <h2 id="what-does-ldap-look-like">What does LDAP look like</h2> <p>LDAP is a tree of objects. Each object has a name, or an RDN (Relative Distinguished Name). The object itself has many key: value pairs in it's data field. If we visualise this, it looks like this.</p> <p><img src="/_static/graph-basic-6.svg" alt="image" /></p> <p>We have the RDN (our tree node's key value), displayed by type=value, and then a set of attributes (the data of the node).</p> <h2 id="naming-things">Naming things</h2> <p>With LDAP often we want to directly reference an node in the tree. To do so, we need a way to uniquely reference the nodes as they exist.</p> <p>Unlike our example trees, where each key is likely to be unique. IE node with key A is cannot exist twice in the tree. In ldap it <em>is</em> valid to have a key exist twice, such as ou=People. This raises a challenge. Previously, we could just &quot;look for A&quot;, and we would have what we wanted. But now, we must not only know the RDN, aka key, that we want to retrieve, but the path through the tree from the root to our target node with the RDN.</p> <p>This is done by walking down the tree til we find what we want. Looking at the image above, consider:</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>dc=com </span><span>dc=example,dc=com </span><span>ou=People,dc=example,dc=com </span><span>uid=user,ou=People,dc=example,dc=com </span></code></pre> <p>We can make a Fully Qualified Distinguished Name (FQDN), or just Distinguished Name(DN), by joining the RDN components. For our example, uid=user,ou=People,dc=example,dc=com. This is a unique path through the tree to the node we wish to access.</p> <p>This should explain why LDAP is called a &quot;tree&quot;, why objects are named the way they are, and help you to visualise the layout of data in your own tree.</p> GDB: Using memory watch points Sat, 11 Jun 2016 00:00:00 +0000 Unknown https://fy.blackhats.net.au/blog/2016-06-11-gdb-using-memory-watch-points/ https://fy.blackhats.net.au/blog/2016-06-11-gdb-using-memory-watch-points/ <h1 id="gdb-using-memory-watch-points">GDB: Using memory watch points</h1> <p>While programming, we've all seen it.</p> <p>&quot;Why the hell is that variable set to 1? It should be X!&quot;</p> <p>A lot of programmers would stack print statements around till the find the issue. Others, might look at function calls.</p> <p>But in the arsenal of the programmer, is the debugger. Normally, the debugger, is really overkill, and too complex to really solve a lot of issues. But while trying to find an issue like this, it shines.</p> <p>All the code we are about to discuss is <a href="https://github.com/Firstyear/liblfdb">in the liblfdb git</a></p> lock free database Tue, 07 Jun 2016 00:00:00 +0000 Unknown https://fy.blackhats.net.au/blog/2016-06-07-lock-free-database/ https://fy.blackhats.net.au/blog/2016-06-07-lock-free-database/ <h1 id="lock-free-database">lock free database</h1> <p>While discussing some ideas with the owner of <a href="http://liblfds.org/">liblfds</a> I was thinking about some of the issues in the database of Directory Server, and other ldap products. What slows us down?</p> <h2 id="why-are-locks-slow">Why are locks slow?</h2> <p>It's a good idea to read <a href="http://liblfds.org/mediawiki/index.php?title=Article:Memory_Barriers">this article</a> to understand Memory Barriers in a cpu.</p> <p>When you think about the way a mutex has to work, it takes advantage of these primitives to create a full barrier, and do the compare and exchange to set the value of the lock, and to guarantee the other memory is synced to our core. This is pretty full on for cpu time, and in reverse, to unlock you have to basically do the same again. That's a lot of operations! (NOTE: You do a load barrier on the way in to the lock, and a store barrier on the unlock. The end result is the full barrier over the set of operations as a whole.)</p> <p>Lock contenion is really the killer though. If every thread is blocked on a single lock, they cannot progress. Given the cost of our lock, if we are stalling our threads behind a lock, we have cpus waiting to do nothing during this process. The OS scheduler helps mask this by waking and running another thread, but eventually contenion will win out.</p> <p>If we bring NUMA into the picture, our mutex may be in a different NUMA region than the thread requesting the lock. This adds an impact to latency as well!</p> <p>We need to try and avoid these operations if we can, to increase performance.</p> <h2 id="bdb">BDB</h2> <p>Currently, BDB basically serialises all operations over a set of locks to access to the data.</p> <p>This means that a set of readers and writers will execute <em>in order</em>, with only one at a time.</p> <p>Not so great for performance, but great for consistency. We are hit hard by our locks, and we have issues with NUMA, especially accessing the page caches, as we regularly have to cross NUMA regions to access the required memory.</p> <h2 id="lmdb">LMDB</h2> <p>LMDB does somewhat better. This is based on a COW btree, with reference counting to accessors. There are still locks scattered through out the tree, which will have an impact however.</p> <p>But, LMDB can establish a read only transaction, of which there can be many, and a serialised, single write transaction. These still suffer from synchronisation of the locks across cores, because LMDB basically allows direct memory access to the tree.</p> <p>As well, NUMA is an issue again: Across a NUMA region, if you access the DB over one of the boundaries, you will suffer a large performance hit. LMDB tries to offset this through it's use of the VFS for page cache. However that's just passing the buck. You now rely on the kernel to have the memory pages needed in the correct region, and that's not guaranteed.</p> <h2 id="what-can-we-do">What can we do?</h2> <p>We need to think about how CPU and RAM works. We need to avoid crossing NUMA regions, and we need to avoid costly CPU instructions and behaviour. We need to avoid contenion on single locks no matter where they may be. We need our program to act less like a human reasons about it, and more like how a CPU works: Asynchronously, and with interprocessor communication. When our program behaviour matches the hardware, we can achieve better performance, and correctness.</p> <p>In the testing of lfds, the lfds author has found that Single Thread, accessing memory within one NUMA region, and without locks, always wins by operation count. This is compared to even lock free behaviours across many cores.</p> <p>This is because in a single thread we don't <em>need</em> to lock data. It has exclusive access, and does not need to contend with other cores. No mutexes needed, no barriers needed.</p> <p>So we must minimise our interprocessor traversal if we can, but we want to keep data into a single CPU region. Our data should ideally be in the NUMA region we want to access it in, in the end.</p> <h2 id="async-db">Async db</h2> <p><em>Disclaimer</em>: This is just an idea, and still needs polish.</p> <p>We run our database, (be it lmdb, bdb, or something new) in a single thread, on one cpu. Now that we are within a single CPU, we can dispense all locking mechanisms, and still have a guarantee that the view of the data is correct.</p> <p>Every thread of our application would then be &quot;pinned&quot; to a NUMA region and core, to ensure that they don't move.</p> <p>We would then use the <a href="http://liblfds.org/mediawiki/index.php?title=r7.1.0:Queue_%28bounded,_single_producer,_single_consumer%29">single producer, single consumer bounded queue</a> from this article. Each NUMA region would establish one of these queues to the database thread. The bounding size is the number of working threads on the system. Each queue would be thread max for the bound, even though there are multiple regions. This is because there may be an unequal distribution of threads to regions, so we may have all threads in one region.</p> <p>Now, our database thread can essentially round robbin, and dequeue requests as they enter. We can use the DB without locks, because we are serialised within one thread. The results would then be placed back into the queue, and the requestor of the operation would be able to examine the results. Because we put the results into the memory space of the requestor, we pay the NUMA price once as we retrieve the result, rather than repeatedly like we do now where we access various caches and allocations.</p> <p>Why would this be better?</p> <ul> <li>We have dispensed completely with ALL mutexes and locks. The queue in liblfds is fast. Amazingly fast. Seriously, look at <a href="http://liblfds.org/mediawiki/index.php?title=r7.1.0:Queue_%28unbounded,_many_producer,_many_consumer%29#Benchmark_Results_and_Analysis">these benchmarks</a>. And that's on the many many queue, which is theoretically slower than the single single bounded queue.</li> <li>We keep consistency within the database. Because we only have one thread acting on the data, we have gained serialisation implicitly.</li> <li>We keep data in the NUMA regions where it needs to be. Rather than having a large cache that spans potentially many NUMA regions, we get data as we need, and put it into the numa region of the DB thread.</li> <li>Because the data is within a single thread, we take advantage of the cpu cache heavily, without the expense of the cpu caches to the other threads. Minimising page evictions and inclusions is a good thing.</li> </ul> <p>There are many other potential ways to improve this:</p> <ul> <li>We could potentially cache entries into the NUMA region. When a search is requested, provided the serial of the entry hasn't been advanced, the entry still within our NUMA region is valid. This prevents more moving between NUMA regions, again yielding a benefit. It would basically be:</li> </ul> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span> Thread A Thread DB </span><span> Queue a read transaction with </span><span> thread ID X --&gt; </span><span> &lt;-- Open the transaction, and stash </span><span> </span><span> Queue a search for set of IDs --&gt; </span><span> Dequeue search request </span><span> Build ID set </span><span> With each ID, pair to the &quot;db version&quot; of the entry. IE USN style. </span><span> &lt;-- Return ID set to queue </span><span> Examine the local cache. </span><span> if ID not in local cache || </span><span> not USN matches entry in cache </span><span> Add Id to &quot;retrieve set&quot; </span><span> Queue a retrieve request --&gt; </span><span> Dequeue the retrieve request </span><span> Copy the requested IDs / entries to Thread A </span><span> &lt;-- Return </span><span> </span><span> Queue a transaction complete --&gt; </span><span> &lt;-- Done </span></code></pre> <h2 id="acknowledgement">Acknowledgement</h2> <p>Huge thank you to Winterflaw, the author of LibLFDS for discussing these ideas, his future review of this idea, and for teaching me many of these concepts.</p> Zero Outage Migration Of Directory Server Infrastructure Fri, 03 Jun 2016 00:00:00 +0000 Unknown https://fy.blackhats.net.au/blog/2016-06-03-zero-outage-migration-of-directory-server-infrastructure/ https://fy.blackhats.net.au/blog/2016-06-03-zero-outage-migration-of-directory-server-infrastructure/ <h1 id="zero-outage-migration-of-directory-server-infrastructure">Zero Outage Migration Of Directory Server Infrastructure</h1> <p>In my previous job I used to manage the Directory Servers for a University. People used to challenge me that while doing migrations, downtime was needed.</p> <p><em>They are all wrong</em></p> <p>It is very possible, and achievable to have zero outage migrations of Directory Servers. All it takes is some thought, planning, dedication and testing.</p> Acis for group creation and delegataion in DS Wed, 25 May 2016 00:00:00 +0000 Unknown https://fy.blackhats.net.au/blog/2016-05-25-acis-for-group-creation-and-delegataion-in-ds/ https://fy.blackhats.net.au/blog/2016-05-25-acis-for-group-creation-and-delegataion-in-ds/ <h1 id="acis-for-group-creation-and-delegataion-in-ds">Acis for group creation and delegataion in DS</h1> <p>Something I get asked about frequently is ACI's in Directory Server. They are a little bit of an art, and they have a lot of edge cases that you can hit.</p> <p>I am asked about &quot;how do I give access to uid=X to create groups in some ou=Y&quot;.</p> <p>TL;DR: You want the ACI at the end of the post. All the others are insecure in some way.</p> <p>So lets do it.</p> <p>First, I have my user:</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>dn: uid=test,ou=People,dc=example,dc=com </span><span>objectClass: top </span><span>objectClass: account </span><span>objectClass: simpleSecurityObject </span><span>uid: test </span><span>userPassword: {SSHA}LQKDZWFI1cw6EnnYtv74v622aPeNZ9cxXc/QIA== </span></code></pre> <p>And I have the ou I want them to edit:</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>dn: ou=custom,ou=Groups,dc=example,dc=com </span><span>objectClass: top </span><span>objectClass: organizationalUnit </span><span>ou: custom </span></code></pre> <p>So I would put the aci onto ou=custom,ou=Groups,dc=example,dc=com, and away I go:</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>aci: (target = &quot;ldap:///ou=custom,ou=groups,dc=example,dc=com&quot;) </span><span> (version 3.0; acl &quot;example&quot;; allow (read, search, write, add) </span><span> (userdn = &quot;ldap:///uid=test,ou=People,dc=example,dc=com&quot;); </span><span> ) </span></code></pre> <p>Great! Now I can add a group under ou=custom,ou=Groups,dc=example,dc=com!</p> <p>But this ACI is REALLY BAD AND INSECURE. Why?</p> <p>First, it allows uid=test,ou=People,dc=example,dc=com write access to the ou=custom itself, which means that it can alter the aci, and potentially grant further rights. That's bad.</p> <p>So lets tighten that up.</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>aci: (target = &quot;ldap:///cn=*,ou=custom,ou=groups,dc=example,dc=com&quot;) </span><span> (version 3.0; acl &quot;example&quot;; allow (read, search, write, add) </span><span> (userdn = &quot;ldap:///uid=test,ou=People,dc=example,dc=com&quot;); </span><span> ) </span></code></pre> <p>Better! Now we can only create objects with cn=* under that ou, and we can't edit the ou or it's aci's itself. But this is still insecure! Imagine, that I made:</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>dn: cn=fake_root,ou=custom,ou=groups,dc=example,dc=com </span><span>.... </span><span>uid: xxxx </span><span>userClass: secure </span><span>memberOf: cn=some_privileged_group,.... </span></code></pre> <p>Many sites often have their pam_ldap/nslcd/sssd set to search from the <em>root</em> IE dc=example,dc=com. Because ldap doesn't define a <em>sort</em> order of responses, this entry may over-ride an exist admin user, or it could be a new user that matches authorisation filters. This just granted someone in your org access to all your servers!</p> <p>But we can prevent this.</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>aci: (target = &quot;ldap:///cn=*,ou=custom,ou=groups,dc=example,dc=com&quot;) </span><span> (targetfilter=&quot;(&amp;(objectClass=top)(objectClass=groupOfUniqueNames))&quot;) </span><span> (version 3.0; acl &quot;example&quot;; allow (read, search, write, add) </span><span> (userdn = &quot;ldap:///uid=test,ou=People,dc=example,dc=com&quot;); </span><span> ) </span></code></pre> <p>Looks better! Now we can only create objects with objectClass top, and groupOfUniqueNames.</p> <p>Then again ....</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>dn: cn=bar,ou=custom,ou=Groups,dc=example,dc=com </span><span>objectClass: top </span><span>objectClass: groupOfUniqueNames </span><span>objectClass: simpleSecurityObject </span><span>cn: bar </span><span>userPassword: {SSHA}UYVTFfPFZrN01puFXYJM3nUcn8lQcVSWtJmQIw== </span></code></pre> <p>Just because we say it has to have top and groupOfUniqueNames DOESN'T exclude adding more objectClasses!</p> <p>Finally, if we make the delegation aci:</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span># This is the secure aci </span><span>aci: (target = &quot;ldap:///cn=*,ou=custom,ou=groups,dc=example,dc=com&quot;) </span><span> (targetfilter=&quot;(&amp;(objectClass=top)(objectClass=groupOfUniqueNames))&quot;) </span><span> (targetattr=&quot;cn || uniqueMember || objectclass&quot;) </span><span> (version 3.0; acl &quot;example&quot;; allow (read, search, write, add) </span><span> (userdn = &quot;ldap:///uid=test,ou=People,dc=example,dc=com&quot;); </span><span> ) </span></code></pre> <p>This aci limits creation to <em>only</em> groups of unique names and top, and limits the attributes to only what can be made in those objectClasses. <em>Finally</em> we have a secure aci. Even though we can add other objectClasses, we can never actually add the attributes to satisfy them, so we effectively limit this to the types show. Even if we add other objectClasses that take &quot;may&quot; as the attribute, we can never fill in those attributes either.</p> <p>Summary: Acis are hard.</p> systemd is not monolithic Mon, 23 May 2016 00:00:00 +0000 Unknown https://fy.blackhats.net.au/blog/2016-05-23-systemd-is-not-monolithic/ https://fy.blackhats.net.au/blog/2016-05-23-systemd-is-not-monolithic/ <h1 id="systemd-is-not-monolithic">systemd is not monolithic</h1> <p>Go ahead. Please read <a href="http://0pointer.de/blog/projects/the-biggest-myths.html">this post by Lennart about systemd myths</a>. I'll wait.</p> <p>Done? Great. You noticed the first point. &quot;Systemd is monolithic&quot;. Which is carefully &quot;debunked&quot;.</p> <p>So this morning while building Ds, I noticed my compile failed:</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>configure: checking for Systemd... </span><span>checking for --with-systemd... using systemd native features </span><span>checking for --with-journald... using journald logging: WARNING, this may cause system instability </span><span>checking for pkg-config... (cached) /usr/bin/pkg-config </span><span>checking for Systemd with pkg-config... configure: error: no Systemd / Journald pkg-config files </span><span>Makefile:84: recipe for target &#39;ds-configure&#39; failed </span></code></pre> <p>I hadn't changed this part of the code, and it's been reliably compiling for me ... What changed?</p> <p>Well on RHEL7 here is the layout of the system libraries:</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>/usr/lib64/libsystemd-daemon.so </span><span>/usr/lib64/libsystemd-id128.so </span><span>/usr/lib64/libsystemd-journal.so </span><span>/usr/lib64/libsystemd-login.so </span><span>/usr/lib64/libsystemd.so </span><span>/usr/lib64/libudev.so </span></code></pre> <p>They also each come with their own very nice pkg-config file so you can find them.</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>/usr/lib64/pkgconfig/libsystemd-daemon.pc </span><span>/usr/lib64/pkgconfig/libsystemd-id128.pc </span><span>/usr/lib64/pkgconfig/libsystemd-journal.pc </span><span>/usr/lib64/pkgconfig/libsystemd-login.pc </span><span>/usr/lib64/pkgconfig/libsystemd.pc </span><span>/usr/lib64/pkgconfig/libudev.pc </span></code></pre> <p>Sure these are big libraries, but it's pretty modular. And it's nice they are seperate out.</p> <p>But today, I compiled on rawhide. What's changed:</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>/usr/lib64/libsystemd.so </span><span>/usr/lib64/libudev.so </span><span> </span><span>/usr/lib64/pkgconfig/libsystemd.pc </span><span>/usr/lib64/pkgconfig/libudev.pc </span></code></pre> <p>I almost thought this was an error. Surely they put libsystemd-journald into another package.</p> <p>No. No they did not.</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>I0&gt; readelf -Ws /usr/lib64/libsystemd.so | grep -i journal_print </span><span> 297: 00000000000248c0 177 FUNC GLOBAL DEFAULT 12 sd_journal_print@@LIBSYSTEMD_209 </span><span> 328: 0000000000024680 564 FUNC GLOBAL DEFAULT 12 sd_journal_printv@@LIBSYSTEMD_209 </span><span> 352: 0000000000023d80 788 FUNC GLOBAL DEFAULT 12 sd_journal_printv_with_location@@LIBSYSTEMD_209 </span><span> 399: 00000000000240a0 162 FUNC GLOBAL DEFAULT 12 sd_journal_print_with_location@@LIBSYSTEMD_209 </span></code></pre> <p>So we went from these small modular libraries:</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>-rwxr-xr-x. 1 root root 26K May 12 14:29 /usr/lib64/libsystemd-daemon.so.0.0.12 </span><span>-rwxr-xr-x. 1 root root 21K May 12 14:29 /usr/lib64/libsystemd-id128.so.0.0.28 </span><span>-rwxr-xr-x. 1 root root 129K May 12 14:29 /usr/lib64/libsystemd-journal.so.0.11.5 </span><span>-rwxr-xr-x. 1 root root 56K May 12 14:29 /usr/lib64/libsystemd-login.so.0.9.3 </span><span>-rwxr-xr-x. 1 root root 159K May 12 14:29 /usr/lib64/libsystemd.so.0.6.0 </span></code></pre> <p>To this monolithic library:</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>-rwxr-xr-x. 1 root root 556K May 22 14:09 /usr/lib64/libsystemd.so.0.15.0 </span></code></pre> <p>&quot;Systemd is not monolithic&quot;.</p> 389ds on freebsd update Sun, 17 Apr 2016 00:00:00 +0000 Unknown https://fy.blackhats.net.au/blog/2016-04-17-389ds-on-freebsd-update/ https://fy.blackhats.net.au/blog/2016-04-17-389ds-on-freebsd-update/ <h1 id="389ds-on-freebsd-update">389ds on freebsd update</h1> <p>A few months ago I posted an how to build 389-ds for freebsd. This was to start my porting effort.</p> <p>I have now finished the port. There are still issues that the perl setup-ds.pl installer does not work, but this will be resolved soon in other ways.</p> <p>For now here are the steps for 389-ds on freebsd. You may need to wait for a few days for the relevant patches to be in git.</p> <p>You will need to install these deps:</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>autotools </span><span>git </span><span>openldap-client </span><span>db5 </span><span>cyrus-sasl </span><span>pkgconf </span><span>nspr </span><span>nss </span><span>net-snmp </span><span>gmake </span><span>python34 </span></code></pre> <p>You will need to use pip for these python dependencies.</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>sudo python3.4 -m ensurepip </span><span>sudo pip3.4 install six pyasn1 pyasn1-modules </span></code></pre> <p>You will need to install svrcore.</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>fetch http://www.port389.org/binaries/svrcore-4.1.1.tar.bz2 </span><span>tar -xvzf svrcore-4.1.1.tar.bz2 </span><span>cd svrcore-4.1.1 </span><span>CFLAGS=&quot;-fPIC &quot;./configure --prefix=/opt/svrcore </span><span>make </span><span>sudo make install </span></code></pre> <p>You will need the following python tools checked out:</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>git clone https://git.fedorahosted.org/git/389/lib389.git </span><span>git clone https://github.com/pyldap/pyldap.git </span><span>cd pyldap </span><span>python3.4 setup.py build </span><span>sudo python3.4 setup.py install </span></code></pre> <p>Now you can clone ds and try to build it:</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>cd </span><span>git clone https://git.fedorahosted.org/git/389/ds.git </span><span>cd ds </span><span>./configure --prefix=/opt/dirsrv --with-openldap=/usr/local --with-db --with-db-inc=/usr/local/include/db5/ --with-db-lib=/usr/local/lib/db5/ --with-sasl --with-sasl-inc=/usr/local/include/sasl/ --with-sasl-lib=/usr/local/lib/sasl2/ --with-svrcore-inc=/opt/svrcore/include/ --with-svrcore-lib=/opt/svrcore/lib/ --with-netsnmp=/usr/local </span><span>gmake </span><span>sudo gmake install </span></code></pre> <p>Go back to the lib389 directory:</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>sudo pw user add dirsrv </span><span>sudo PYTHONPATH=`pwd` python3.4 lib389/clitools/ds_setup.py -f ~/setup-ds-admin.inf -v </span><span>sudo chown -R dirsrv:dirsrv /opt/dirsrv/var/{run,lock,log,lib}/dirsrv </span><span>sudo chmod 775 /opt/dirsrv/var </span><span>sudo chmod 775 /opt/dirsrv/var/* </span><span>sudo /opt/dirsrv/sbin/ns-slapd -d 0 -D /opt/dirsrv/etc/dirsrv/slapd-localhost </span></code></pre> <p>This is a really minimal setup routine right now. If it all worked, you can now run your instance. Here is my output belowe:</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>INFO:lib389.tools:Running setup with verbose </span><span>INFO:lib389.tools:Using inf from /home/william/setup-ds-admin.inf </span><span>INFO:lib389.tools:Configuration [&#39;general&#39;, &#39;slapd&#39;, &#39;rest&#39;, &#39;backend-userRoot&#39;] </span><span>INFO:lib389.tools:Configuration general {&#39;selinux&#39;: False, &#39;full_machine_name&#39;: &#39;localhost.localdomain&#39;, &#39;config_version&#39;: 2, &#39;strict_host_checking&#39;: True} </span><span>INFO:lib389.tools:Configuration slapd {&#39;secure_port&#39;: 636, &#39;root_password&#39;: &#39;password&#39;, &#39;port&#39;: 389, &#39;cert_dir&#39;: &#39;/opt/dirsrv/etc/dirsrv/slapd-localhost/&#39;, &#39;lock_dir&#39;: &#39;/opt/dirsrv/var/lock/dirsrv/slapd-localhost&#39;, &#39;ldif_dir&#39;: &#39;/opt/dirsrv/var/lib/dirsrv/slapd-localhost/ldif&#39;, &#39;backup_dir&#39;: &#39;/opt/dirsrv/var/lib/dirsrv/slapd-localhost/bak&#39;, &#39;prefix&#39;: &#39;/opt/dirsrv&#39;, &#39;instance_name&#39;: &#39;localhost&#39;, &#39;bin_dir&#39;: &#39;/opt/dirsrv/bin/&#39;, &#39;data_dir&#39;: &#39;/opt/dirsrv/share/&#39;, &#39;local_state_dir&#39;: &#39;/opt/dirsrv/var&#39;, &#39;run_dir&#39;: &#39;/opt/dirsrv/var/run/dirsrv&#39;, &#39;schema_dir&#39;: &#39;/opt/dirsrv/etc/dirsrv/slapd-localhost/schema&#39;, &#39;config_dir&#39;: &#39;/opt/dirsrv/etc/dirsrv/slapd-localhost/&#39;, &#39;root_dn&#39;: &#39;cn=Directory Manager&#39;, &#39;log_dir&#39;: &#39;/opt/dirsrv/var/log/dirsrv/slapd-localhost&#39;, &#39;tmp_dir&#39;: &#39;/tmp&#39;, &#39;user&#39;: &#39;dirsrv&#39;, &#39;group&#39;: &#39;dirsrv&#39;, &#39;db_dir&#39;: &#39;/opt/dirsrv/var/lib/dirsrv/slapd-localhost/db&#39;, &#39;sbin_dir&#39;: &#39;/opt/dirsrv/sbin&#39;, &#39;sysconf_dir&#39;: &#39;/opt/dirsrv/etc&#39;, &#39;defaults&#39;: &#39;1.3.5&#39;} </span><span>INFO:lib389.tools:Configuration backends [{&#39;name&#39;: &#39;userRoot&#39;, &#39;sample_entries&#39;: True, &#39;suffix&#39;: &#39;dc=example,dc=com&#39;}] </span><span>INFO:lib389.tools:PASSED: user / group checking </span><span>INFO:lib389.tools:PASSED: Hostname strict checking </span><span>INFO:lib389.tools:PASSED: prefix checking </span><span>INFO:lib389:dir (sys) : /opt/dirsrv/etc/sysconfig </span><span>INFO:lib389.tools:PASSED: instance checking </span><span>INFO:lib389.tools:PASSED: root user checking </span><span>INFO:lib389.tools:PASSED: network avaliability checking </span><span>INFO:lib389.tools:READY: beginning installation </span><span>INFO:lib389.tools:ACTION: creating /opt/dirsrv/var/lib/dirsrv/slapd-localhost/bak </span><span>INFO:lib389.tools:ACTION: creating /opt/dirsrv/etc/dirsrv/slapd-localhost/ </span><span>INFO:lib389.tools:ACTION: creating /opt/dirsrv/etc/dirsrv/slapd-localhost/ </span><span>INFO:lib389.tools:ACTION: creating /opt/dirsrv/var/lib/dirsrv/slapd-localhost/db </span><span>INFO:lib389.tools:ACTION: creating /opt/dirsrv/var/lib/dirsrv/slapd-localhost/ldif </span><span>INFO:lib389.tools:ACTION: creating /opt/dirsrv/var/lock/dirsrv/slapd-localhost </span><span>INFO:lib389.tools:ACTION: creating /opt/dirsrv/var/log/dirsrv/slapd-localhost </span><span>INFO:lib389.tools:ACTION: creating /opt/dirsrv/var/run/dirsrv </span><span>INFO:lib389.tools:ACTION: Creating certificate database is /opt/dirsrv/etc/dirsrv/slapd-localhost/ </span><span>INFO:lib389.tools:ACTION: Creating dse.ldif </span><span>INFO:lib389.tools:FINISH: completed installation </span><span>Sucessfully created instance </span><span> </span><span>[17/Apr/2016:14:44:21.030683607 +1000] could not open config file &quot;/opt/dirsrv/etc/dirsrv/slapd-localhost//slapd-collations.conf&quot; - absolute path? </span><span>[17/Apr/2016:14:44:21.122087994 +1000] 389-Directory/1.3.5.1 B2016.108.412 starting up </span><span>[17/Apr/2016:14:44:21.460033554 +1000] convert_pbe_des_to_aes: Checking for DES passwords to convert to AES... </span><span>[17/Apr/2016:14:44:21.461012440 +1000] convert_pbe_des_to_aes - No DES passwords found to convert. </span><span>[17/Apr/2016:14:44:21.462712083 +1000] slapd started. Listening on All Interfaces port 389 for LDAP requests </span></code></pre> <p>If we do an ldapsearch:</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>fbsd-389-port# uname -r -s </span><span>FreeBSD 10.2-RELEASE </span><span>fbsd-389-port# ldapsearch -h localhost -b &#39;&#39; -s base -x + </span><span># extended LDIF </span><span># </span><span># LDAPv3 </span><span># base &lt;&gt; with scope baseObject </span><span># filter: (objectclass=*) </span><span># requesting: + </span><span># </span><span> </span><span># </span><span>dn: </span><span>creatorsName: cn=server,cn=plugins,cn=config </span><span>modifiersName: cn=server,cn=plugins,cn=config </span><span>createTimestamp: 20160417044112Z </span><span>modifyTimestamp: 20160417044112Z </span><span>subschemaSubentry: cn=schema </span><span>supportedExtension: 2.16.840.1.113730.3.5.7 </span><span>supportedExtension: 2.16.840.1.113730.3.5.8 </span><span>supportedExtension: 1.3.6.1.4.1.4203.1.11.3 </span><span>supportedExtension: 1.3.6.1.4.1.4203.1.11.1 </span><span>supportedControl: 2.16.840.1.113730.3.4.2 </span><span>supportedControl: 2.16.840.1.113730.3.4.3 </span><span>supportedControl: 2.16.840.1.113730.3.4.4 </span><span>supportedControl: 2.16.840.1.113730.3.4.5 </span><span>supportedControl: 1.2.840.113556.1.4.473 </span><span>supportedControl: 2.16.840.1.113730.3.4.9 </span><span>supportedControl: 2.16.840.1.113730.3.4.16 </span><span>supportedControl: 2.16.840.1.113730.3.4.15 </span><span>supportedControl: 2.16.840.1.113730.3.4.17 </span><span>supportedControl: 2.16.840.1.113730.3.4.19 </span><span>supportedControl: 1.3.6.1.1.13.1 </span><span>supportedControl: 1.3.6.1.1.13.2 </span><span>supportedControl: 1.3.6.1.4.1.42.2.27.8.5.1 </span><span>supportedControl: 1.3.6.1.4.1.42.2.27.9.5.2 </span><span>supportedControl: 1.2.840.113556.1.4.319 </span><span>supportedControl: 1.3.6.1.4.1.42.2.27.9.5.8 </span><span>supportedControl: 1.3.6.1.4.1.4203.666.5.16 </span><span>supportedControl: 2.16.840.1.113730.3.4.14 </span><span>supportedControl: 2.16.840.1.113730.3.4.20 </span><span>supportedControl: 1.3.6.1.4.1.1466.29539.12 </span><span>supportedControl: 2.16.840.1.113730.3.4.12 </span><span>supportedControl: 2.16.840.1.113730.3.4.18 </span><span>supportedFeatures: 1.3.6.1.4.1.4203.1.5.1 </span><span>supportedSASLMechanisms: EXTERNAL </span><span>supportedLDAPVersion: 2 </span><span>supportedLDAPVersion: 3 </span><span>vendorName: 389 Project </span><span>vendorVersion: 389-Directory/1.3.5.1 B2016.108.412 </span></code></pre> The future vision of 389-ds Sat, 16 Apr 2016 00:00:00 +0000 Unknown https://fy.blackhats.net.au/blog/2016-04-16-the-future-vision-of-389-ds/ https://fy.blackhats.net.au/blog/2016-04-16-the-future-vision-of-389-ds/ <h1 id="the-future-vision-of-389-ds">The future vision of 389-ds</h1> <p>Disclaimer: This is my vision and analysis of 389-ds and it's future. It is nothing about Red Hat's future plans or goals. Like all predictions, they may not even come true.</p> <p>As I have said before I'm part of the 389-ds core team. I really do have a passion for authentication and identity management: I'm sure some of my friends would like to tell me to shut up about it sometimes.</p> <p>389-ds, or rather, ns-slapd has a lot of history. Originally from the umich code base, it has moved through Netscape, SUN, Aol and, finally to Red Hat. It's quite something to find myself working on code that was written in 1996. In 1996 I was on a playground in Primary School, where my biggest life concerns was if I could see the next episode of [anime of choice] the next day at before school care. What I'm saying, is ns-slapd is old. Very old. There are many dark, untrodden paths in that code base.</p> <p>You would get your nice big iron machine from SUN, you would setup the ns-slapd instance once. You would then probably setup one other ns-slapd master, then you would run them in production for the next 4 to 5 years with minimal changes. Business policy would ask developers to integrate with the LDAP server. Everything was great.</p> <p>But it's not 1996 anymore. I have managed to complete schooling and acquire a degree in this time. ns-slapd has had many improvements, but the work flow and way that ns-slapd in managed really hasn't changed a lot.</p> <p>While ns-slapd has stayed roughly the same, the world has evolved. We now have latte sipping code hipsters, sitting in trendy Melbourne cafes programming in go and whatever js framework of this week. They deploy to AWS, to GCE. (But not Azure, that's not cool enough). These developers have a certain mindset and the benefits of centralised business authentication isn't one of them. They want to push things to cloud, but no system administrator would let a corporate LDAP be avaliable on the internet. CIO's are all drinking the &quot;cloud&quot; &quot;disruption&quot; kool aid. The future of many technologies is certainly in question.</p> <p>To me, there is no doubt that ns-slapd is still a great technology: Like the unix philosohpy, tools should do &quot;one thing&quot; and &quot;one thing well&quot;. When it comes to secure authentication, user identification, and authorisation, LDAP is still king. So why are people not deploying it in their new fancy containers and cloud disruption train that they are all aboard?</p> <p>ns-slapd is old. Our systems and installers, such as setup-ds.pl are really designed for the &quot;pet&quot; mentality of servers. They are hard to automate to replica groups, and they inist on having certain types of information avaliable before they can run. They also don't work with automation, and are unable to accept certain types of ldifs as part of the inf file that drives the install. You have to have at least a few years experience with ns-slapd before you could probably get this process &quot;right&quot;.</p> <p>Another, well, LDAP is ... well, hard. It's not json (which is apparently the only thing developers understand now). Developers also don't care about identifying users. That's just not <em>cool</em>. Why would we try and use some &quot;hard&quot; LDAP system, when I can just keep some json in a mongodb that tracks your password and groups you are in?</p> <p>So what can we do? Where do I see 389-ds going in the future?</p> <ul> <li>We need to modernise our tooling, and installers. It needs to be easier than ever to setup an LDAP instance. Our administration needs to move away from applying ldifs, into robust, command line tools.</li> <li>Setting up replication groups and masters needs to be simpler. Replication topologies should be &quot;self managing&quot; (to an extent). Ie I should say &quot;here is a new ldap server, join this replication group&quot;. The administration layer then determines all the needed replication agreements for robust and avaliable service.</li> <li>We need to get away from long lived static masters, and be able to have rapidly deployed, and destroyed, masters. With the changes above, this will lend itself to faster and easier deployment into containers and other such systems.</li> <li>During updates, we need to start to enable smarter choices by default: but still allow people to fix their systems on certain configurations to guarantee stability. For example, we add new options and improvements to DS all the time: but we cannot always enable them by default. This makes our system look dated, when really a few configurations would really modernise and help improve deployments. Having mechanisms to push the updates to clients who want it, and enable them by default on new installs will go a long way.</li> <li>Out of the box we need smarter settings: The <em>default</em> install should need almost <em>no changes</em> to be a strong, working LDAP system. It should not require massive changes or huge amounts of indepth knowledge to deploy. I'm the LDAP expert: You're the coffee sipping developer. You should be able to trust the defaults we give you, and know that they will be well engineered and carefully considered.</li> <li>Finally I think what is also going to be really important is Web Based authentication. We need to provide ways to setup and provision SAML and OAuth systems that &quot;just work&quot; with our LDAP. With improvements on the way like <a href="https://tools.ietf.org/html/draft-wibrown-ldapssotoken-00">this draft rfc</a> will even allow fail over between token systems, backed by the high security and performance guarantees of LDAP.</li> </ul> <p>This is my vision of the future for 389-ds: Simplification of setup. Polish of the configuration. Ability to automate and tools to empower administrators. Integration with what developers want.</p> <p>Lets see how much myself and the team can achieve by the end of 2016.</p> Disabling journald support Thu, 14 Apr 2016 00:00:00 +0000 Unknown https://fy.blackhats.net.au/blog/2016-04-14-disabling-journald-support/ https://fy.blackhats.net.au/blog/2016-04-14-disabling-journald-support/ <h1 id="disabling-journald-support">Disabling journald support</h1> <p>Some people may have noticed that there is a feature open for Directory Server to support journald.</p> <p>As of April 13th, we have decided to disable support for this in Directory Server.</p> <p>This isn't because anyone necesarrily hates or dislikes systemd. All too often people discount systemd due to a hate reflex.</p> <p>This decision came about due to known, hard, technical limitations of journald. This is not hand waving opinion, this is based on testing, numbers, business requirements and experience.</p> <p>So lets step back for a second. Directory Server is an LDAP server. On a network, LDAP is deployed typically to be responsible for authentication and authorisation of users to services. This is a highly security sensitive role. This leads to an important facet of security being auditability. For example, the need to track when and who has authenticated to a network. The ability to audit what permisions were requested and granted. Further more, the ability to audit and identify <em>changes</em> to Directory Server data which may represent a compromise or change of user credential or authorisation rights.</p> <p>Being able to audit these is of vital importance, from small buisinesses to large enterprise. As a security system, this audit trail must have guarantees of <em>correctness</em> and <em>avaliablility</em>. Often a business will have internal rules around the length of time auditing information must be retained for. In other businesses there are legal requirements for auditing information to be retained for long periods. Often a business will keep in excess of 2 weeks of authentication and authorisation data for the purposes of auditing.</p> <p>Directory Server provides this auditing capability through it's logging functions. Directory Server is configured to produce 3 log types.</p> <ul> <li>errors - Contains Directory Server operations, plugin data, changes. This is used by administrators to identify service behaviour and issues.</li> <li>access - Contains a log of all search and bind (authentication) operations.</li> <li>audit - Contains a log of all modifications, additions and deletions of objects within the Directory Server.</li> </ul> <p>For the purpose of auditing in a security context the access and audit logs are of vital importance, as is their retention times.</p> <p>So why is journald not fit for purpose in this context? It seems to be fine for many other systems?</p> <p>Out of the box, journald has a <em>hardcoded</em> limit on the maximum capacity of logs. This is 4GB of on disk capacity. Once this is exceeded, the journal rotates, and begins to overwrite entries at the beginging of the log. Think circular buffer. After testing and identifying the behaviours of Directory Server, and the size of journald messages, I determined that a medium to large site will cause the journal to begin a rotation in 3 hours or less during high traffic periods.</p> <p>3 hours is a far smaller number than the &quot;weeks&quot; of retention that is required for auditing purposes of most businesses.</p> <p>Additionally, by default journald is configured to drop events if they enter the log to rapidly. This is a &quot;performance&quot; enhancement. However, during my tests I found that 85% of Directory Server events were being dropped. This violates the need for correct and complete audit logs in a security system.</p> <p>This <em>can</em> be reconfigured, but the question should be asked. Why are log events dropped at all? On a system, log events are the basis of auditing and accountability, forming a historical account of evidence for an Administrator or Security personel to trace in the event of an incident. Dropping events from Directory Server <em>is unacceptable</em>. As I stated, this can be reconfigured.</p> <p>But it does begin to expose the third point. Performance. Journald is slow, and caused an increase of 15% cpu and higher IO on my testing environments. For a system such as Directory Server, this overhead is unacceptable. We consider performance impacts of 2% to be signifigant: We cannot accept 15%.</p> <p>As an API journald is quite nice, and has many useful features. However, we as a team cannot support journald with these three limitations above.</p> <p>If journald support is to be taken seriously by security and performance sensitive applications the following changes are recomended.</p> <ul> <li>Remove the 4G log size limit. It can either be configurable by a user, or there should be no limits.</li> <li>Log events should either not be dropped by default, or a method to have per systemd unit file overrides to prevent dropping of certain services events should be added.</li> <li>The performance of journald should be improved as to reduce the impact upon applications consuming the journald api.</li> </ul> <p>I hope that this explains why we have decided to remove systemd's journald support from Directory Server at this time.</p> <p>Before I am asked: No I will not reverse my stance on this matter, and I will continue to advise my team of the same. Systemd needs to come to the table and improve their api before we can consider it for use.</p> <p>The upstream issue can be seen here <a href="https://fedorahosted.org/389/ticket/47968">389 ds trac 47968</a>. All of my calculations are in this thread too.</p> Enabling the 389 ds nightly builds Thu, 14 Apr 2016 00:00:00 +0000 Unknown https://fy.blackhats.net.au/blog/2016-04-14-enabling-the-389-ds-nightly-builds/ https://fy.blackhats.net.au/blog/2016-04-14-enabling-the-389-ds-nightly-builds/ <h1 id="enabling-the-389-ds-nightly-builds">Enabling the 389 ds nightly builds</h1> <p>I maintain a copr repo which I try to keep update with &quot;nightly&quot; builds of 389-ds.</p> <p>You can use the following to enable them for EL7:</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>sudo -s </span><span>cd /etc/yum.repos.d </span><span>wget https://copr.fedorainfracloud.org/coprs/firstyear/ds/repo/epel-7/firstyear-ds-epel-7.repo </span><span>wget https://copr.fedorainfracloud.org/coprs/firstyear/svrcore/repo/epel-7/firstyear-svrcore-epel-7.repo </span><span>wget https://copr.fedorainfracloud.org/coprs/firstyear/rest389/repo/epel-7/firstyear-rest389-epel-7.repo </span><span>wget https://copr.fedorainfracloud.org/coprs/firstyear/lib389/repo/epel-7/firstyear-lib389-epel-7.repo </span><span>yum install python-lib389 python-rest389 389-ds-base </span></code></pre> 389 ds aci linting tool Fri, 01 Apr 2016 00:00:00 +0000 Unknown https://fy.blackhats.net.au/blog/2016-04-01-389-ds-aci-linting-tool/ https://fy.blackhats.net.au/blog/2016-04-01-389-ds-aci-linting-tool/ <h1 id="389-ds-aci-linting-tool">389 ds aci linting tool</h1> <p>In the past I have discussed aci's and their management in directory server.</p> <p>It's a very complex topic, and there are issues that can arise.</p> <p>I have now created an aci linting tool which can connect to a directory server and detect common mistakes in acis, along with explinations of how to correct them.</p> <p>This will be in a release of lib389 in the future. For now, it's under review and hopefully will be accepted soon!</p> <p>Here is sample output below.</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>------------------------------------------------------------------------------- </span><span>Directory Server Aci Lint Error: DSALE0001 </span><span>Severity: HIGH </span><span> </span><span>Affected Acis: </span><span>(targetattr!=&quot;userPassword&quot;)(version 3.0; acl &quot;Enable anonymous access&quot;; allow (read, search, compare) userdn=&quot;ldap:///anyone&quot;;) </span><span>(targetattr !=&quot;cn || sn || uid&quot;)(targetfilter =&quot;(ou=Accounting)&quot;)(version 3.0;acl &quot;Accounting Managers Group Permissions&quot;;allow (write)(groupdn = &quot;ldap:///cn=Accounting Managers,ou=groups,dc=example,dc=com&quot;);) </span><span>(targetattr !=&quot;cn || sn || uid&quot;)(targetfilter =&quot;(ou=Human Resources)&quot;)(version 3.0;acl &quot;HR Group Permissions&quot;;allow (write)(groupdn = &quot;ldap:///cn=HR Managers,ou=groups,dc=example,dc=com&quot;);) </span><span>(targetattr !=&quot;cn ||sn || uid&quot;)(targetfilter =&quot;(ou=Product Testing)&quot;)(version 3.0;acl &quot;QA Group Permissions&quot;;allow (write)(groupdn = &quot;ldap:///cn=QA Managers,ou=groups,dc=example,dc=com&quot;);) </span><span>(targetattr !=&quot;cn || sn || uid&quot;)(targetfilter =&quot;(ou=Product Development)&quot;)(version 3.0;acl &quot;Engineering Group Permissions&quot;;allow (write)(groupdn = &quot;ldap:///cn=PD Managers,ou=groups,dc=example,dc=com&quot;);) </span><span> </span><span>Details: </span><span>An aci of the form &quot;(targetAttr!=&quot;attr&quot;)&quot; exists on your system. This aci </span><span>will internally be expanded to mean &quot;all possible attributes including system, </span><span>excluding the listed attributes&quot;. </span><span> </span><span>This may allow access to a bound user or anonymous to read more data about </span><span>directory internals, including aci state or user limits. In the case of write </span><span>acis it may allow a dn to set their own resource limits, unlock passwords or </span><span>their own aci. </span><span> </span><span>The ability to change the aci on the object may lead to privilege escalation in </span><span>some cases. </span><span> </span><span> </span><span>Advice: </span><span>Convert the aci to the form &quot;(targetAttr=&quot;x || y || z&quot;)&quot;. </span><span> </span><span>------------------------------------------------------------------------------- </span><span>Directory Server Aci Lint Error: DSALE0002 </span><span>Severity: HIGH </span><span> </span><span>Affected Acis: </span><span>ou=People,dc=example,dc=com (targetattr !=&quot;cn || sn || uid&quot;)(targetfilter =&quot;(ou=Accounting)&quot;)(version 3.0;acl &quot;Accounting Managers Group Permissions&quot;;allow (write)(groupdn = &quot;ldap:///cn=Accounting Managers,ou=groups,dc=example,dc=com&quot;);) </span><span>|- ou=People,dc=example,dc=com (targetattr !=&quot;cn || sn || uid&quot;)(targetfilter =&quot;(ou=Human Resources)&quot;)(version 3.0;acl &quot;HR Group Permissions&quot;;allow (write)(groupdn = &quot;ldap:///cn=HR Managers,ou=groups,dc=example,dc=com&quot;);) </span><span>|- ou=People,dc=example,dc=com (targetattr !=&quot;cn ||sn || uid&quot;)(targetfilter =&quot;(ou=Product Testing)&quot;)(version 3.0;acl &quot;QA Group Permissions&quot;;allow (write)(groupdn = &quot;ldap:///cn=QA Managers,ou=groups,dc=example,dc=com&quot;);) </span><span>|- ou=People,dc=example,dc=com (targetattr !=&quot;cn || sn || uid&quot;)(targetfilter =&quot;(ou=Product Development)&quot;)(version 3.0;acl &quot;Engineering Group Permissions&quot;;allow (write)(groupdn = &quot;ldap:///cn=PD Managers,ou=groups,dc=example,dc=com&quot;);) </span><span> </span><span>ou=People,dc=example,dc=com (targetattr !=&quot;cn || sn || uid&quot;)(targetfilter =&quot;(ou=Human Resources)&quot;)(version 3.0;acl &quot;HR Group Permissions&quot;;allow (write)(groupdn = &quot;ldap:///cn=HR Managers,ou=groups,dc=example,dc=com&quot;);) </span><span>|- ou=People,dc=example,dc=com (targetattr !=&quot;cn || sn || uid&quot;)(targetfilter =&quot;(ou=Accounting)&quot;)(version 3.0;acl &quot;Accounting Managers Group Permissions&quot;;allow (write)(groupdn = &quot;ldap:///cn=Accounting Managers,ou=groups,dc=example,dc=com&quot;);) </span><span>|- ou=People,dc=example,dc=com (targetattr !=&quot;cn ||sn || uid&quot;)(targetfilter =&quot;(ou=Product Testing)&quot;)(version 3.0;acl &quot;QA Group Permissions&quot;;allow (write)(groupdn = &quot;ldap:///cn=QA Managers,ou=groups,dc=example,dc=com&quot;);) </span><span>|- ou=People,dc=example,dc=com (targetattr !=&quot;cn || sn || uid&quot;)(targetfilter =&quot;(ou=Product Development)&quot;)(version 3.0;acl &quot;Engineering Group Permissions&quot;;allow (write)(groupdn = &quot;ldap:///cn=PD Managers,ou=groups,dc=example,dc=com&quot;);) </span><span> </span><span>ou=People,dc=example,dc=com (targetattr !=&quot;cn ||sn || uid&quot;)(targetfilter =&quot;(ou=Product Testing)&quot;)(version 3.0;acl &quot;QA Group Permissions&quot;;allow (write)(groupdn = &quot;ldap:///cn=QA Managers,ou=groups,dc=example,dc=com&quot;);) </span><span>|- ou=People,dc=example,dc=com (targetattr !=&quot;cn || sn || uid&quot;)(targetfilter =&quot;(ou=Accounting)&quot;)(version 3.0;acl &quot;Accounting Managers Group Permissions&quot;;allow (write)(groupdn = &quot;ldap:///cn=Accounting Managers,ou=groups,dc=example,dc=com&quot;);) </span><span>|- ou=People,dc=example,dc=com (targetattr !=&quot;cn || sn || uid&quot;)(targetfilter =&quot;(ou=Human Resources)&quot;)(version 3.0;acl &quot;HR Group Permissions&quot;;allow (write)(groupdn = &quot;ldap:///cn=HR Managers,ou=groups,dc=example,dc=com&quot;);) </span><span>|- ou=People,dc=example,dc=com (targetattr !=&quot;cn || sn || uid&quot;)(targetfilter =&quot;(ou=Product Development)&quot;)(version 3.0;acl &quot;Engineering Group Permissions&quot;;allow (write)(groupdn = &quot;ldap:///cn=PD Managers,ou=groups,dc=example,dc=com&quot;);) </span><span> </span><span>ou=People,dc=example,dc=com (targetattr !=&quot;cn || sn || uid&quot;)(targetfilter =&quot;(ou=Product Development)&quot;)(version 3.0;acl &quot;Engineering Group Permissions&quot;;allow (write)(groupdn = &quot;ldap:///cn=PD Managers,ou=groups,dc=example,dc=com&quot;);) </span><span>|- ou=People,dc=example,dc=com (targetattr !=&quot;cn || sn || uid&quot;)(targetfilter =&quot;(ou=Accounting)&quot;)(version 3.0;acl &quot;Accounting Managers Group Permissions&quot;;allow (write)(groupdn = &quot;ldap:///cn=Accounting Managers,ou=groups,dc=example,dc=com&quot;);) </span><span>|- ou=People,dc=example,dc=com (targetattr !=&quot;cn || sn || uid&quot;)(targetfilter =&quot;(ou=Human Resources)&quot;)(version 3.0;acl &quot;HR Group Permissions&quot;;allow (write)(groupdn = &quot;ldap:///cn=HR Managers,ou=groups,dc=example,dc=com&quot;);) </span><span>|- ou=People,dc=example,dc=com (targetattr !=&quot;cn ||sn || uid&quot;)(targetfilter =&quot;(ou=Product Testing)&quot;)(version 3.0;acl &quot;QA Group Permissions&quot;;allow (write)(groupdn = &quot;ldap:///cn=QA Managers,ou=groups,dc=example,dc=com&quot;);) </span><span> </span><span> </span><span>Details: </span><span>Acis on your system exist which are both not equals targetattr, and overlap in </span><span>scope. </span><span> </span><span>The way that directory server processes these, is to invert them to to white </span><span>lists, then union the results. </span><span> </span><span>As a result, these acis *may* allow access to the attributes you want them to </span><span>exclude. </span><span> </span><span>Consider: </span><span> </span><span>aci: (targetattr !=&quot;cn&quot;)(version 3.0;acl &quot;Self write all but cn&quot;;allow (write) </span><span> (userdn = &quot;ldap:///self&quot;);) </span><span>aci: (targetattr !=&quot;sn&quot;)(version 3.0;acl &quot;Self write all but sn&quot;;allow (write) </span><span> (userdn = &quot;ldap:///self&quot;);) </span><span> </span><span>This combination allows self write to *all* attributes within the subtree. </span><span> </span><span>In cases where the target is members of a group, it may allow a member who is </span><span>within two groups to have elevated privilege. </span><span> </span><span> </span><span>Advice: </span><span>Convert the aci to the form &quot;(targetAttr=&quot;x || y || z&quot;)&quot;. </span><span> </span><span>Prevent the acis from overlapping, and have them on unique subtrees. </span><span> </span><span>------------------------------------------------------------------------------- </span><span>FAIL </span></code></pre> Trick to debug single files in ds Wed, 16 Mar 2016 00:00:00 +0000 Unknown https://fy.blackhats.net.au/blog/2016-03-16-trick-to-debug-single-files-in-ds/ https://fy.blackhats.net.au/blog/2016-03-16-trick-to-debug-single-files-in-ds/ <h1 id="trick-to-debug-single-files-in-ds">Trick to debug single files in ds</h1> <p>I've been debugging thread deadlocks in directory server. When you turn on detailed tracing with</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>ns-slapd -d 1 </span></code></pre> <p>You slow the server down so much that you can barely function.</p> <p>A trick is that defines in the local .c file, override from the .h. Copy paste this to the file you want to debug. This allows the logs from this file to be emitted at -d 0, but without turning it on everywhere, so you don't grind the server to a halt.</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>/* Do this so we can get the messages at standard log levels. */ </span><span>#define SLAPI_LOG_FATAL 0 </span><span>#define SLAPI_LOG_TRACE 0 </span><span>#define SLAPI_LOG_PACKETS 0 </span><span>#define SLAPI_LOG_ARGS 0 </span><span>#define SLAPI_LOG_CONNS 0 </span><span>#define SLAPI_LOG_BER 0 </span><span>#define SLAPI_LOG_FILTER 0 </span><span>#define SLAPI_LOG_CONFIG 0 </span><span>#define SLAPI_LOG_ACL 0 </span><span>#define SLAPI_LOG_SHELL 0 </span><span>#define SLAPI_LOG_PARSE 0 </span><span>#define SLAPI_LOG_HOUSE 0 </span><span>#define SLAPI_LOG_REPL 0 </span><span>#define SLAPI_LOG_CACHE 0 </span><span>#define SLAPI_LOG_PLUGIN 0 </span><span>#define SLAPI_LOG_TIMING 0 </span><span>#define SLAPI_LOG_BACKLDBM 0 </span><span>#define SLAPI_LOG_ACLSUMMARY 0 </span></code></pre> Blog migration Wed, 09 Mar 2016 00:00:00 +0000 Unknown https://fy.blackhats.net.au/blog/2016-03-09-blog-migration/ https://fy.blackhats.net.au/blog/2016-03-09-blog-migration/ <h1 id="blog-migration">Blog migration</h1> <p>I've migrated my blog from django to tinkerer. I've also created a number of helper pages to preserve all the links to old pages.</p> <p>Please let me know if anything is wrong using my contact details on the about page.</p> <p>William</p> ldctl to generate test objects Tue, 23 Feb 2016 00:00:00 +0000 Unknown https://fy.blackhats.net.au/blog/2016-02-23-ldctl-to-generate-test-objects/ https://fy.blackhats.net.au/blog/2016-02-23-ldctl-to-generate-test-objects/ <h1 id="ldctl-to-generate-test-objects">ldctl to generate test objects</h1> <p>I was told by some coworkers today at Red Hat that I can infact use ldctl to generate my databases for load testing with 389-ds.</p> <p>First, create a template.ldif</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>objectClass: top </span><span>objectclass: person </span><span>objectClass: organizationalPerson </span><span>objectClass: inetorgperson </span><span>objectClass: posixAccount </span><span>objectClass: shadowAccount </span><span>sn: testnew[A] </span><span>cn: testnew[A] </span><span>uid: testnew[A] </span><span>givenName: testnew[A] </span><span>description: description[A] </span><span>userPassword: testnew[A] </span><span>mail: testnew[A]@redhat.com </span><span>uidNumber: 3[A] </span><span>gidNumber: 4[A] </span><span>shadowMin: 0 </span><span>shadowMax: 99999 </span><span>shadowInactive: 30 </span><span>shadowWarning: 7 </span><span>homeDirectory: /home/uid[A] </span></code></pre> <p>Now you can use ldctl to actually load in the data:</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>ldclt -h localhost -p 389 -D &quot;cn=Directory Manager&quot; -w password -b &quot;ou=people,dc=example,dc=com&quot; \ </span><span>-I 68 -e add,commoncounter -e &quot;object=/tmp/template.ldif,rdn=uid:[A=INCRNNOLOOP(0;3999;5)]&quot; </span></code></pre> <p>Thanks to vashirov and spichugi for their advice and this example!</p> Patches Welcome Wed, 17 Feb 2016 00:00:00 +0000 Unknown https://fy.blackhats.net.au/blog/2016-02-17-patches-welcome/ https://fy.blackhats.net.au/blog/2016-02-17-patches-welcome/ <h1 id="patches-welcome">&quot;Patches Welcome&quot;</h1> <p>&quot;Patches Welcome&quot;. We've all seen it in the Open Source community. Nothing makes me angrier than these two words.</p> <p>Often this is said by people who are too busy, or too lazy to implement features that just aren't of interest to them. This isn't the response you get when you submit a bad idea, or something technically unfeasible. It's the response that speaks of an apathy to your software's users.</p> <p>I get that we all have time limits for development. I know that we have to prioritise. I know that it may not be of import to the business right now. Even at the least, reach out, say you'll create a ticket on their behalf if they cannot. Help them work through the design, then implement it in the future.</p> <p>But do not ever consider yourself so high and mighty that the request of a user &quot;isn't good enough for you&quot;. These are your customers, supporters, advocates, bug reporters, testers, and users. They are what build the community. A community is not just the developers of the software. It's the users of it too, and their skills are separate from those of the the developer.</p> <p>Often people ask for features, but do not have the expertise, or domain knowledge to implement them. That does not invalidate the worth of the feature, if anything speaks to it's value as a real customer will benefit from this, and your project as a whole will improve. Telling them &quot;Patches Welcome&quot; is like saying &quot;I know you aren't capable of implementing this yourself. I don't care to help you at all, and I don't want to waste my time on you. Go away&quot;.</p> <p>As is obvious from this blog, I'm part of the 389 Directory Server Team.</p> <p>I will <em>never</em> tell a user that &quot;patches welcome&quot;. I will always support them to design their idea. I will ask them to lodge a ticket, or I'll do it for them if they cannot. If a user can and wants to try to implement the software of their choice, I will help them and teach them. If they cannot, I will make sure that at some time in the future, we can deliver it to them, or if we cannot, a real, honest explanation of why.</p> <p>That's the community in 389 I am proud to be a part of.</p> Failed to delete old semaphore for stats file Tue, 09 Feb 2016 00:00:00 +0000 Unknown https://fy.blackhats.net.au/blog/2016-02-09-failed-to-delete-old-semaphore-for-stats-file/ https://fy.blackhats.net.au/blog/2016-02-09-failed-to-delete-old-semaphore-for-stats-file/ <h1 id="failed-to-delete-old-semaphore-for-stats-file">Failed to delete old semaphore for stats file</h1> <p>Today I was getting this error:</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>[09/Feb/2016:12:21:26 +101800] - 389-Directory/1.3.5 B2016.040.145 starting up </span><span>[09/Feb/2016:12:21:26 +101800] - Failed to delete old semaphore for stats file (/opt/dirsrv/var/run/dirsrv/slapd-localhost.stats). Error 13 (Permission denied). </span></code></pre> <p>But when you check:</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>/opt# ls -al /opt/dirsrv/var/run/dirsrv/slapd-localhost.stats </span><span>ls: cannot access /opt/dirsrv/var/run/dirsrv/slapd-localhost.stats: No such file or directory </span></code></pre> <p>Turns out on linux this isn't actually where the file is. You need to remove:</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>/dev/shm/sem.slapd-localhost.stats </span></code></pre> <p>A bug will be opened shortly ....</p> Securing IPA Tue, 09 Feb 2016 00:00:00 +0000 Unknown https://fy.blackhats.net.au/blog/2016-02-09-securing-ipa/ https://fy.blackhats.net.au/blog/2016-02-09-securing-ipa/ <h1 id="securing-ipa">Securing IPA</h1> <p><a href="/blog/html/2019/07/10/i_no_longer_recommend_freeipa.html">I no longer recommend using FreeIPA - Read more here!</a></p> <p>By default IPA has some weak security around TLS and anonymous binds.</p> <p>We can improve this by changing the following options.</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>nsslapd-minssf-exclude-rootdse: on </span><span>nsslapd-minssf: 56 </span><span>nsslapd-require-secure-binds: on </span></code></pre> <p>The last one you may want to change is:</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>nsslapd-allow-anonymous-access: on </span></code></pre> <p>I think this is important to have on, as it allows non-domain members to use ipa, but there are arguments to disabling anon reads too.</p> 389 on freebsd Thu, 28 Jan 2016 00:00:00 +0000 Unknown https://fy.blackhats.net.au/blog/2016-01-28-389-on-freebsd/ https://fy.blackhats.net.au/blog/2016-01-28-389-on-freebsd/ <h1 id="389-on-freebsd">389 on freebsd</h1> <p>I've decided to start porting 389-ds to freebsd.</p> <p>So tonight I took the first steps. Let's see if we can get it to build in a dev environment like I would use normally.</p> <p>You will need to install these deps:</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>autotools </span><span>git </span><span>openldap-client </span><span>db5 </span><span>cyrus-sasl </span><span>pkgconf </span><span>nspr </span><span>nss </span><span>net-snmp </span><span>gmake </span></code></pre> <p>You then need to install svrcore. I'll likely add a port for this too.</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>fetch https://ftp.mozilla.org/pub/directory/svrcore/releases/4.0.4/src/svrcore-4.0.4.tar.bz2 </span><span>tar -xvjf svrcore-4.0.4.tar.bz2 </span><span>cd svrcore-4.0.4 </span><span>CFLAGS=&quot;-fPIC &quot;./configure --prefix=/opt/svrcore </span><span>make </span><span>sudo make install </span></code></pre> <p>Now you can clone ds and try to build it:</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>cd </span><span>git clone https://git.fedorahosted.org/git/389/ds.git </span><span>cd ds </span><span>./configure --prefix=/opt/dirsrv --with-openldap=/usr/local --with-db --with-db-inc=/usr/local/include/db5/ --with-db-lib=/usr/local/lib/db5/ --with-sasl --with-sasl-inc=/usr/local/include/sasl/ --with-sasl-lib=/usr/local/lib/sasl2/ --with-svrcore-inc=/opt/svrcore/include/ --with-svrcore-lib=/opt/svrcore/lib/ --with-netsnmp=/usr/local </span><span>gmake </span></code></pre> <p>If it's like me you get the following:</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>make: &quot;/usr/home/admin_local/ds/Makefile&quot; line 10765: warning: duplicate script for target &quot;%/dirsrv&quot; ignored </span><span>make: &quot;/usr/home/admin_local/ds/Makefile&quot; line 10762: warning: using previous script for &quot;%/dirsrv&quot; defined here </span><span>make: &quot;/usr/home/admin_local/ds/Makefile&quot; line 10767: warning: duplicate script for target &quot;%/dirsrv&quot; ignored </span><span>make: &quot;/usr/home/admin_local/ds/Makefile&quot; line 10762: warning: using previous script for &quot;%/dirsrv&quot; defined here </span><span>make: &quot;/usr/home/admin_local/ds/Makefile&quot; line 10768: warning: duplicate script for target &quot;%/dirsrv&quot; ignored </span><span>make: &quot;/usr/home/admin_local/ds/Makefile&quot; line 10762: warning: using previous script for &quot;%/dirsrv&quot; defined here </span><span>perl ./ldap/servers/slapd/mkDBErrStrs.pl -i /usr/local/include/db5/ -o . </span><span>make all-am </span><span>make[1]: &quot;/usr/home/admin_local/ds/Makefile&quot; line 10765: warning: duplicate script for target &quot;%/dirsrv&quot; ignored </span><span>make[1]: &quot;/usr/home/admin_local/ds/Makefile&quot; line 10762: warning: using previous script for &quot;%/dirsrv&quot; defined here </span><span>make[1]: &quot;/usr/home/admin_local/ds/Makefile&quot; line 10767: warning: duplicate script for target &quot;%/dirsrv&quot; ignored </span><span>make[1]: &quot;/usr/home/admin_local/ds/Makefile&quot; line 10762: warning: using previous script for &quot;%/dirsrv&quot; defined here </span><span>make[1]: &quot;/usr/home/admin_local/ds/Makefile&quot; line 10768: warning: duplicate script for target &quot;%/dirsrv&quot; ignored </span><span>make[1]: &quot;/usr/home/admin_local/ds/Makefile&quot; line 10762: warning: using previous script for &quot;%/dirsrv&quot; defined here </span><span>depbase=`echo ldap/libraries/libavl/avl.o | sed &#39;s|[^/]*$|.deps/&amp;|;s|\.o$||&#39;`; cc -DHAVE_CONFIG_H -I. -DBUILD_NUM= -DVENDOR=&quot;\&quot;389 Project\&quot;&quot; -DBRAND=&quot;\&quot;389\&quot;&quot; -DCAPBRAND=&quot;\&quot;389\&quot;&quot; -UPACKAGE_VERSION -UPACKAGE_TARNAME -UPACKAGE_STRING -UPACKAGE_BUGREPORT -I./ldap/include -I./ldap/servers/slapd -I./include -I. -DLOCALSTATEDIR=&quot;\&quot;/opt/dirsrv/var\&quot;&quot; -DSYSCONFDIR=&quot;\&quot;/opt/dirsrv/etc\&quot;&quot; -DLIBDIR=&quot;\&quot;/opt/dirsrv/lib\&quot;&quot; -DBINDIR=&quot;\&quot;/opt/dirsrv/bin\&quot;&quot; -DDATADIR=&quot;\&quot;/opt/dirsrv/share\&quot;&quot; -DDOCDIR=&quot;\&quot;/opt/dirsrv/share/doc/389-ds-base\&quot;&quot; -DSBINDIR=&quot;\&quot;/opt/dirsrv/sbin\&quot;&quot; -DPLUGINDIR=&quot;\&quot;/opt/dirsrv/lib/dirsrv/plugins\&quot;&quot; -DTEMPLATEDIR=&quot;\&quot;/opt/dirsrv/share/dirsrv/data\&quot;&quot; -g -O2 -MT ldap/libraries/libavl/avl.o -MD -MP -MF $depbase.Tpo -c -o ldap/libraries/libavl/avl.o ldap/libraries/libavl/avl.c &amp;&amp; mv -f $depbase.Tpo $depbase.Po </span><span>rm -f libavl.a </span><span>ar cru libavl.a ldap/libraries/libavl/avl.o </span><span>ranlib libavl.a </span><span>cc -DHAVE_CONFIG_H -I. -DBUILD_NUM= -DVENDOR=&quot;\&quot;389 Project\&quot;&quot; -DBRAND=&quot;\&quot;389\&quot;&quot; -DCAPBRAND=&quot;\&quot;389\&quot;&quot; -UPACKAGE_VERSION -UPACKAGE_TARNAME -UPACKAGE_STRING -UPACKAGE_BUGREPORT -I./ldap/include -I./ldap/servers/slapd -I./include -I. -DLOCALSTATEDIR=&quot;\&quot;/opt/dirsrv/var\&quot;&quot; -DSYSCONFDIR=&quot;\&quot;/opt/dirsrv/etc\&quot;&quot; -DLIBDIR=&quot;\&quot;/opt/dirsrv/lib\&quot;&quot; -DBINDIR=&quot;\&quot;/opt/dirsrv/bin\&quot;&quot; -DDATADIR=&quot;\&quot;/opt/dirsrv/share\&quot;&quot; -DDOCDIR=&quot;\&quot;/opt/dirsrv/share/doc/389-ds-base\&quot;&quot; -DSBINDIR=&quot;\&quot;/opt/dirsrv/sbin\&quot;&quot; -DPLUGINDIR=&quot;\&quot;/opt/dirsrv/lib/dirsrv/plugins\&quot;&quot; -DTEMPLATEDIR=&quot;\&quot;/opt/dirsrv/share/dirsrv/data\&quot;&quot; -I./lib/ldaputil -I/usr/local/include -I/usr/local/include/nss -I/usr/local/include/nss/nss -I/usr/local/include/nspr -I/usr/local/include/nspr -g -O2 -MT lib/ldaputil/libldaputil_a-cert.o -MD -MP -MF lib/ldaputil/.deps/libldaputil_a-cert.Tpo -c -o lib/ldaputil/libldaputil_a-cert.o `test -f &#39;lib/ldaputil/cert.c&#39; || echo &#39;./&#39;`lib/ldaputil/cert.c </span><span>In file included from lib/ldaputil/cert.c:16: </span><span>/usr/include/malloc.h:3:2: error: &quot;&lt;malloc.h&gt; has been replaced by &lt;stdlib.h&gt;&quot; </span><span>#error &quot;&lt;malloc.h&gt; has been replaced by &lt;stdlib.h&gt;&quot; </span><span> ^ </span><span>1 error generated. </span><span>*** Error code 1 </span><span> </span><span>Stop. </span><span>make[1]: stopped in /usr/home/admin_local/ds </span><span>*** Error code 1 </span><span> </span><span>Stop. </span><span>make: stopped in /usr/home/admin_local/ds </span></code></pre> <p>Time to start looking at including some #ifdef __FREEBSD__ macros.</p> Renaming ovirt storage targets Sat, 16 Jan 2016 00:00:00 +0000 Unknown https://fy.blackhats.net.au/blog/2016-01-16-renaming-ovirt-storage-targets/ https://fy.blackhats.net.au/blog/2016-01-16-renaming-ovirt-storage-targets/ <h1 id="renaming-ovirt-storage-targets">Renaming ovirt storage targets</h1> <p>I run an ovirt server, and sometimes like a tinker that I am, I like to rename things due to new hardware or other ideas that come up.</p> <p>Ovirt makes it quite hard to change the nfs target or name of a storage volume. Although it's not supported, I'm more than happy to dig through the database.</p> <p>NOTE: Take a backup before you start, this is some serious unsupported magic here.</p> <p>First, we need to look at the main tables that are involved in nfs storage:</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>engine=# select id,storage,storage_name from storage_domain_static; </span><span> id | storage | storage_name </span><span>--------------------------------------+--------------------------------------+------------------- </span><span> 6bffd537-badb-43c9-91b2-a922cf847533 | 842add9e-ffef-44d9-bf6d-4f8231b375eb | def_t2_nfs_import </span><span> c3aa02d8-02fd-4a16-bfe6-59f9348a0b1e | 5b8ba182-7d05-44e4-9d64-2a1bb529b797 | def_t2_nfs_iso </span><span> a8ac8bd0-cf40-45ae-9f39-b376c16b7fec | d2fd5e4b-c3de-4829-9f4a-d56246f5454b | def_t2_nfs_lcs </span><span> d719e5f2-f59d-434d-863e-3c9c31e4c02f | e2ba769c-e5a3-4652-b75d-b68959369b55 | def_t1_nfs_master </span><span> a085aca5-112c-49bf-aa91-fbf59e8bde0b | f5be3009-4c84-4d59-9cfe-a1bcedac4038 | def_t1_nfs_sas </span><span> </span><span>engine=# select id,connection from storage_server_connections; </span><span> id | connection </span><span>--------------------------------------+----------------------------------------------------------------- </span><span> 842add9e-ffef-44d9-bf6d-4f8231b375eb | mion.ipa.example.com:/var/lib/exports/t2/def_t2_nfs_import </span><span> 5b8ba182-7d05-44e4-9d64-2a1bb529b797 | mion.ipa.example.com:/var/lib/exports/t2/def_t2_nfs_iso </span><span> d2fd5e4b-c3de-4829-9f4a-d56246f5454b | mion.ipa.example.com:/var/lib/exports/t2/def_t2_nfs_lcs </span><span> e2ba769c-e5a3-4652-b75d-b68959369b55 | mion.ipa.example.com:/var/lib/exports/t1/def_t1_nfs_master </span><span> f5be3009-4c84-4d59-9cfe-a1bcedac4038 | mion.ipa.example.com:/var/lib/exports/t1/def_t1_nfs_sas </span></code></pre> <p>So we are going to rename the def_t2_nfs targets to def_t3_nfs. First we need to update the mount point:</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>update storage_server_connections set connection=&#39;mion.ipa.example.com:/var/lib/exports/t3/def_t3_nfs_import&#39; where id=&#39;842add9e-ffef-44d9-bf6d-4f8231b375eb&#39;; </span><span> </span><span>update storage_server_connections set connection=&#39;mion.ipa.example.com:/var/lib/exports/t3/def_t3_nfs_iso&#39; where id=&#39;5b8ba182-7d05-44e4-9d64-2a1bb529b797&#39;; </span><span> </span><span>update storage_server_connections set connection=&#39;mion.ipa.example.com:/var/lib/exports/t2/def_t2_nfs_lcs&#39; where id=&#39;d2fd5e4b-c3de-4829-9f4a-d56246f5454b&#39;; </span></code></pre> <p>Next we are going to replace the name in the storage_domain_static table.</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>update storage_domain_static set storage_name=&#39;def_t3_nfs_lcs&#39; where storage=&#39;d2fd5e4b-c3de-4829-9f4a-d56246f5454b&#39;; </span><span> </span><span>update storage_domain_static set storage_name=&#39;def_t3_nfs_iso&#39; where storage=&#39;5b8ba182-7d05-44e4-9d64-2a1bb529b797&#39;; </span><span> </span><span>update storage_domain_static set storage_name=&#39;def_t3_nfs_import&#39; where storage=&#39;842add9e-ffef-44d9-bf6d-4f8231b375eb&#39;; </span></code></pre> <p>That's it! Now check it all looks correct and restart.</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>engine=# select id,storage,storage_name from storage_domain_static; </span><span> id | storage | storage_name </span><span>--------------------------------------+--------------------------------------+------------------- </span><span> a8ac8bd0-cf40-45ae-9f39-b376c16b7fec | d2fd5e4b-c3de-4829-9f4a-d56246f5454b | def_t3_nfs_lcs </span><span> c3aa02d8-02fd-4a16-bfe6-59f9348a0b1e | 5b8ba182-7d05-44e4-9d64-2a1bb529b797 | def_t3_nfs_iso </span><span> 6bffd537-badb-43c9-91b2-a922cf847533 | 842add9e-ffef-44d9-bf6d-4f8231b375eb | def_t3_nfs_import </span><span> d719e5f2-f59d-434d-863e-3c9c31e4c02f | e2ba769c-e5a3-4652-b75d-b68959369b55 | def_t1_nfs_master </span><span> a085aca5-112c-49bf-aa91-fbf59e8bde0b | f5be3009-4c84-4d59-9cfe-a1bcedac4038 | def_t1_nfs_sas </span><span>(5 rows) </span><span> </span><span>engine=# select id,connection from storage_server_connections; </span><span> id | connection </span><span>--------------------------------------+----------------------------------------------------------------- </span><span> e2ba769c-e5a3-4652-b75d-b68959369b55 | mion.ipa.example.com:/var/lib/exports/t1/def_t1_nfs_master </span><span> f5be3009-4c84-4d59-9cfe-a1bcedac4038 | mion.ipa.example.com:/var/lib/exports/t1/def_t1_nfs_sas </span><span> 842add9e-ffef-44d9-bf6d-4f8231b375eb | mion.ipa.example.com:/var/lib/exports/t3/def_t3_nfs_import </span><span> 5b8ba182-7d05-44e4-9d64-2a1bb529b797 | mion.ipa.example.com:/var/lib/exports/t3/def_t3_nfs_iso </span><span> d2fd5e4b-c3de-4829-9f4a-d56246f5454b | mion.ipa.example.com:/var/lib/exports/t3/def_t3_nfs_lcs </span><span>(5 rows) </span></code></pre> Running your own mailserver: Mailbox rollover Fri, 15 Jan 2016 00:00:00 +0000 Unknown https://fy.blackhats.net.au/blog/2016-01-15-running-your-own-mailserver-mailbox-rollover/ https://fy.blackhats.net.au/blog/2016-01-15-running-your-own-mailserver-mailbox-rollover/ <h1 id="running-your-own-mailserver-mailbox-rollover">Running your own mailserver: Mailbox rollover</h1> <p>UPDATE 2019: Don't run your own! Use fastmail instead :D!</p> <p>I go to a lot of effort to run my own email server. I don't like google, and I want to keep them away from my messages. While it incurs both financial, and administrative cost, sometimes the benefits are fantastic.</p> <p>I like to sort my mail to folders based on server side filters (which are fantastic, server side filtering is the way to go). I also like to keep my mailboxes in yearly fashion, so they don't grow tooo large. I keep every email I ever receive, and it's saved my arse a few times.</p> <p>Rolling over year to year for most people would be a pain: You need to move all the emails from one folder (mailbox) to another, which incurs a huge time / download / effort cost.</p> <p>Running your own mailserver though, you don't have this issue. It takes a few seconds to complete a year rollover. You can even script it like I did.</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>#!/bin/bash </span><span> </span><span>export MAILUSER=&#39;email address here&#39; </span><span>export LASTYEAR=&#39;2015&#39; </span><span>export THISYEAR=&#39;2016&#39; </span><span> </span><span># Stop postfix first. this way server side filters aren&#39;t being used and mails routed while we fiddle around. </span><span>systemctl stop postfix </span><span> </span><span># Now we can fiddle with mailboxes </span><span> </span><span># First, we want to make the new archive. </span><span> </span><span>doveadm mailbox create -u ${MAILUSER} archive.${THISYEAR} </span><span> </span><span># Create a list of mailboxes. </span><span> </span><span>export MAILBOXES=`doveadm mailbox list -u ${MAILUSER} &#39;INBOX.*&#39; | awk -F &#39;.&#39; &#39;{print $2}&#39;` </span><span>echo $MAILBOXES </span><span> </span><span># Now move the directories to archive. </span><span># Create the new equivalents </span><span> </span><span>for MAILBOX in ${MAILBOXES} </span><span>do </span><span> doveadm mailbox rename -u ${MAILUSER} INBOX.${MAILBOX} archive.${LASTYEAR}.${MAILBOX} </span><span> doveadm mailbox subscribe -u ${MAILUSER} archive.${LASTYEAR}.${MAILBOX} </span><span> doveadm mailbox create -u ${MAILUSER} INBOX.${MAILBOX} </span><span>done </span><span> </span><span>doveadm mailbox list -u ${MAILUSER} </span><span> </span><span># Start postfix back up </span><span> </span><span>systemctl start postfix </span></code></pre> <p>Now I have clean, shiny mailboxes, all my filters still work, and my previous year's emails are tucked away for safe keeping and posterity.</p> <p>The only catch with my script is you need to run it on January 1st, else you get 2016 mails in the 2015 archive. You also still need to move the inbox contents from 2015 manually to the archive. But it's not nearly the same hassle as moving thousands of mailing list messages around.</p> FreeRADIUS: Using mschapv2 with freeipa Wed, 13 Jan 2016 00:00:00 +0000 Unknown https://fy.blackhats.net.au/blog/2016-01-13-freeradius-using-mschapv2-with-freeipa/ https://fy.blackhats.net.au/blog/2016-01-13-freeradius-using-mschapv2-with-freeipa/ <h1 id="freeradius-using-mschapv2-with-freeipa">FreeRADIUS: Using mschapv2 with freeipa</h1> <p><a href="/blog/html/2019/07/10/i_no_longer_recommend_freeipa.html">I no longer recommend using FreeIPA - Read more here!</a></p> <p>Wireless and radius is pretty much useless without mschapv2 and peap. This is because iPhones, androids, even linux have fundamental issues with ttls or other 802.1x modes. mschapv2 &quot;just works&quot;, yet it's one of the most obscure to get working in some cases without AD.</p> <p>If you have an active directory environment, it's pretty well a painless process. But when you want to use anything else, you are in a tight spot.</p> <p>The FreeRADIUS team go on <em>a lot</em> about how mschapv2 doesn't work with ldap: and they are correct. mschapv2 is a challenge response protocol, and you can't do that in conjunction with an ldap bind.</p> <p>However it <em>IS</em> possible to use mschapv2 with an ldap server: It's just not obvious or straight forwards.</p> <p>The way that this works is you need freeradius to look up a user to an ldap dn, then you read (not bind) the nthash of the user from their dn. From there, the FreeRADIUS server is able to conduct the challenge response component.</p> <p>So the main things here to note:</p> <ul> <li>nthash are pretty much an md4. They are broken and terrible. But you need to use them, so you need to secure the access to these.</li> <li>Because you need to secure these, you need to be sure your access controls are correct.</li> </ul> <p>We can pretty easily make this setup work with freeipa in fact.</p> <p>First, follow the contents of <a href="/blog/html/2015/07/06/FreeIPA:_Giving_permissions_to_service_accounts..html">my previous blog post</a> on how to setup the adtrust components and the access controls.</p> <p>You don't actually need to complete the trust with AD, you just need to run the setup util, as this triggers IPA to generate and store nthashes in ipaNTHash on the user account.</p> <p>Now armed with your service account that can read these hashes, and the password, we need to configure FreeRADIUS.</p> <p>FreeRADIUS is EXTREMELY HARD TO CONFIGURE. You can mess it up VERY QUICKLY.</p> <p>Thankfully, the developers provide an excellent default configuration that should only need minimal tweaks to make this configuration work.</p> <p>first, symlink ldap to mods-enabled</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>cd /etc/raddb/mods-enabled </span><span>ln -s ../mods-available/ldap ./ldap </span></code></pre> <p>Now, edit the ldap config in mods-available (That way if a swap file is made, it's not put into mods-enabled where it may do damage)</p> <p>You need to change the parameters to match your site, however the most important setting is:</p> <p>:</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>identity = krbprincipalname=radius/host.ipa.example.net.au@IPA.EXAMPLE.NET.AU,cn=services,cn=accounts,dc=ipa,dc=example,dc=net,dc=au </span><span>password = SERVICE ACCOUNT BIND PW </span><span> </span><span>....snip..... </span><span> </span><span>update { </span><span> ....snip...... </span><span> control:NT-Password·· := &#39;ipaNTHash&#39; </span><span>} </span><span> </span><span> .....snip .... </span><span> </span><span>user { </span><span> base_dn = &quot;cn=users,cn=accounts,dc=ipa,dc=example,dc=net,dc=au&quot; </span><span> filter = &quot;(uid=%{%{Stripped-User-Name}:-%{User-Name}})&quot; </span><span> ....snip.... </span><span>} </span></code></pre> <p>Next, you want to edit the mods-available/eap</p> <p>you want to change the value of default_eap_type to:</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>default_eap_type = mschapv2 </span></code></pre> <p>Finally, you need to update your sites-available, most likely inner-tunnel and default to make sure that they contain:</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>authorize { </span><span> </span><span> ....snip ..... </span><span> -ldap </span><span> </span><span>} </span></code></pre> <p>That's it! Now you should be able to test an ldap account with radtest, using the default NAS configured in /etc/raddb/clients.conf.</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>radtest -t mschap william password 127.0.0.1:1812 0 testing123 </span><span> User-Name = &#39;william&#39; </span><span> NAS-IP-Address = 172.24.16.13 </span><span> NAS-Port = 0 </span><span> Message-Authenticator = 0x00 </span><span> MS-CHAP-Challenge = 0x642690f62148e238 </span><span> MS-CHAP-Response = .... </span><span>Received Access-Accept Id 130 from 127.0.0.1:1812 to 127.0.0.1:56617 length 84 </span><span> MS-CHAP-MPPE-Keys = 0x </span><span> MS-MPPE-Encryption-Policy = Encryption-Allowed </span><span> MS-MPPE-Encryption-Types = RC4-40or128-bit-Allowed </span></code></pre> <h2 id="why-not-use-krb">Why not use KRB?</h2> <p>I was asked in IRC about using KRB keytabs for authenticating the service account. Now the configuration is quite easy - but I won't put it hear.</p> <p>The issue is that it opens up a number of weaknesses. Between FreeRADIUS and LDAP you have communication. Now FreeIPA/389DS doesn't allow GSSAPI over LDAPS/StartTLS. When you are doing an MSCHAPv2 authentication this isn't so bad: FreeRADIUS authenticates with GSSAPI with encryption layers, then reads the NTHash. The NTHash is used inside FreeRADIUS to generate the challenge, and the 802.1x authentication suceeds or fails.</p> <p>Now what happens when we use PAP instead? FreeRADIUS can either read the NTHash and do a comparison (as above), or it can <em>directly bind</em> to the LDAP server. This means in the direct bind case, that the transport <em>may not be encrypted</em> due to the keytab. See, the keytab when used for the service account, will install encryption, but when the simple bind occurs, we don't have GSSAPI material, so we would send this clear text.</p> <p>Which one will occur ... Who knows! FreeRADIUS is a complex piece of software, as is LDAP. Unless you are willing to test all the different possibilities of 802.1x types and LDAP interactions, there is a risk here.</p> <p>Today the only secure, guaranteed way to protect your accounts is TLS. You should use LDAPS, and this guarantees all communication will be secure. It's simpler, faster, and better.</p> <p>That's why I don't document or advise how to use krb keytabs with this configuration.</p> <p>Thanks to <em>moep</em> for helping point out some of the issues with KRB integration.</p> db2index: entry too large (X bytes) for the buffer size (Y bytes) Thu, 17 Dec 2015 00:00:00 +0000 Unknown https://fy.blackhats.net.au/blog/2015-12-17-db2index-entry-too-large-x-bytes-for-the-buffer-size-y-bytes/ https://fy.blackhats.net.au/blog/2015-12-17-db2index-entry-too-large-x-bytes-for-the-buffer-size-y-bytes/ <h1 id="db2index-entry-too-large-x-bytes-for-the-buffer-size-y-bytes">db2index: entry too large (X bytes) for the buffer size (Y bytes)</h1> <p>We've been there: You need to reindex your dirsrv and get it back into production as fast as you can. Then all of a sudden you get this error.</p> <p>Some quick research shows no way to change the mystical buffer size being referenced. You pull out your hair and wonder what's going on, so you play with some numbers, and eventually it works, but you don't know why.</p> <p>It turns out, this is one of the more magical undocumented values that DS sets for itself. If we look through the code, we find that this buffer is derived from the ldbm instances c_maxsize.</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>./ldap/servers/slapd/back-ldbm/import.c:48: </span><span> </span><span>job-&gt;fifo.bsize = (inst-&gt;inst_cache.c_maxsize/10) &lt;&lt; 3; </span></code></pre> <p>That c_maxsize is actually the value of cn=config,cn=ldbm database,cn=plugins,cn=config, nsslapd-dbcachesize.</p> <p>So, say that we get the error bytes is too small as it's only (20000000 bytes) in size. We plug this in:</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>(20000000 &gt;&gt; 3) * 10 = 25000000 </span></code></pre> <p>Which in my case was the size of nsslapd-dbcachesize</p> <p>If we have a hypothetical value, say 28000000 bytes, and db2index can't run, you can use this reverse to calculate the dbcachesize you need:</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>(28000000 &gt;&gt; 3) * 10 = 35000000 </span></code></pre> <p>This will create a buffersize of 28000000 so you can run the db2index task.</p> <p>In the future, this value will be configurable, rather than derived which will improve the clarity of the error, and the remediation.</p> Load balanced 389 instance with freeipa kerberos domain. Fri, 11 Dec 2015 00:00:00 +0000 Unknown https://fy.blackhats.net.au/blog/2015-12-11-load-balanced-389-instance-with-freeipa-kerberos-domain/ https://fy.blackhats.net.au/blog/2015-12-11-load-balanced-389-instance-with-freeipa-kerberos-domain/ <h1 id="load-balanced-389-instance-with-freeipa-kerberos-domain">Load balanced 389 instance with freeipa kerberos domain.</h1> <p><a href="/blog/html/2019/07/10/i_no_longer_recommend_freeipa.html">I no longer recommend using FreeIPA - Read more here!</a></p> <p>First, create a fake host that we can assign services too. This is for the load balancer (f5, netscaler, ace, haproxy)</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>ipa host-add haproxydemo.ipa.example.com --random --force </span></code></pre> <p>Now you can add the keytab for the loadbalanced service.</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>ipa service-add --force ldap/haproxydemo.ipa.example.com </span></code></pre> <p>Then you need to delegate the keytab to the ldap servers that will sit behind the lb.</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>ipa service-add-host ldap/haproxydemo.ipa.example.com --hosts=liza.ipa.example.com </span></code></pre> <p>You should be able to extract this keytab on the host now.</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>ipa-getkeytab -s alina.ipa.example.com -p ldap/haproxydemo.ipa.example.com -k /etc/dirsrv/slapd-localhost/ldap.keytab </span></code></pre> <p>into /etc/sysconfig/dirsrv-localhost</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>KRB5_KTNAME=/etc/dirsrv/slapd-localhost/ldap.keytab </span></code></pre> <p>Now, restart the instance and make sure you can't connect directly.</p> <p>Setup haproxy. I had a huge amount of grief with ipv6, so I went v4 only for this demo. :</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>global </span><span> log 127.0.0.1 local2 </span><span> </span><span> chroot /var/lib/haproxy </span><span> pidfile /var/run/haproxy.pid </span><span> maxconn 4000 </span><span> user haproxy </span><span> group haproxy </span><span> daemon </span><span> </span><span> stats socket /var/lib/haproxy/stats </span><span> </span><span>listen ldap :3389 </span><span> mode tcp </span><span> balance roundrobin </span><span> </span><span> server ldap 10.0.0.2:389 check </span><span> timeout connect 10s </span><span> timeout server 1m </span><span> </span><span>ldapsearch -H ldap://haproxydemo.ipa.example.com:3389 -Y GSSAPI </span></code></pre> <p>Reveals a working connection!</p> Debbuging and patching 389-ds. Tue, 08 Dec 2015 00:00:00 +0000 Unknown https://fy.blackhats.net.au/blog/2015-12-08-debbuging-and-patching-389-ds/ https://fy.blackhats.net.au/blog/2015-12-08-debbuging-and-patching-389-ds/ <h1 id="debbuging-and-patching-389-ds">Debbuging and patching 389-ds.</h1> <p>Debugging and working on software like 389-ds looks pretty daunting. However, I think it's one of the easiest projects to setup, debug and contribute to (for a variety of reasons).</p> <p>Fixing issues like the one referenced in this post is a good way to get your hands dirty into C, gdb, and the project in general. It's how I started, by solving small issues like this, and working up to managing larger fixes and commits. You will end up doing a lot of research and testing, but you learn a lot for it.</p> <p>Additionally, the 389-ds team are great people, and very willing to help and walk you through debugging and issue solving like this.</p> <p>Lets get started!</p> <p>First, lets get your build env working.</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>git clone http://git.fedorahosted.org/git/389/ds.git </span></code></pre> <p>If you need to apply any patches to test, now is the time:</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>cd ds </span><span>git am ~/path/to/patch </span></code></pre> <p>Now we can actually get all the dependencies. Changes these paths to suit your environment.</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>export DSPATH=~/development/389ds/ds </span><span>sudo yum-builddep 389-ds-base </span><span>sudo yum install libasan llvm </span><span>mkdir -p ~/build/ds/ </span><span>cd ~/build/ds/ &amp;&amp; $DSPATH/configure --with-openldap --enable-debug --enable-asan --prefix=/opt/dirsrv/ </span><span>make -C ~/build/ds </span><span>sudo make -C ~/build/ds install </span></code></pre> <p>NOTE: Thanks to Viktor for the tip about yum-builddep working without a spec file.</p> <p>If you are still missing packages, these commands are rough, but work.</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>sudo yum install `grep &quot;^BuildRequires&quot; $DSPATH/rpm/389-ds-base.spec.in | awk &#39;{ print $2 }&#39; | grep -v &quot;^/&quot;` </span><span>sudo yum install `grep &quot;^Requires:&quot; $DSPATH/ds/rpm/389-ds-base.spec.in | awk &#39;{ print $2 $3 $4 $5 $6 $7 }&#39; | grep -v &quot;^/&quot; | grep -v &quot;name&quot;` </span></code></pre> <p>Now with that out the way, we can get into it. Setup the ds install:</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>sudo /opt/dirsrv/sbin/setup-ds.pl --debug General.StrictHostChecking=false </span></code></pre> <p>If you have enabled ASAN you may notice that the install freezes trying to start slapd. That's okay, at this point you can control C it. If setup-ds.pl finishes, even better.</p> <p>Now lets run the instance up:</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>sudo -s </span><span>export ASAN_SYMBOLIZER_PATH=/usr/bin/llvm-symbolizer </span><span>export ASAN_OPTIONS=symbolize=1 </span><span>/opt/dirsrv/sbin/ns-slapd -d 0 -D /opt/dirsrv/etc/dirsrv/slapd-localhost </span><span> </span><span>[08/Dec/2015:13:09:01 +1000] - 389-Directory/1.3.5 B2015.342.252 starting up </span><span>================================================================= </span><span>==28682== ERROR: AddressSanitizer: unknown-crash on address 0x7fff49a54ff0 at pc 0x7f59bc0f719f bp 0x7fff49a54c80 sp 0x7fff49a54c28 </span></code></pre> <p>Uh oh! We have a crash. Lets work it out.</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>================================================================= </span><span>==28682== ERROR: AddressSanitizer: unknown-crash on address 0x7fff49a54ff0 at pc 0x7f59bc0f719f bp 0x7fff49a54c80 sp 0x7fff49a54c28 </span><span>WRITE of size 513 at 0x7fff49a54ff0 thread T0 </span><span> #0 0x7f59bc0f719e in scanf_common /usr/src/debug/gcc-4.8.3-20140911/obj-x86_64-redhat-linux/x86_64-redhat-linux/libsanitizer/asan/../../../../libsanitizer/sanitizer_common/sanitizer_common_interceptors_scanf.inc:305 </span><span> #1 0x7f59bc0f78b6 in __interceptor_vsscanf /usr/src/debug/gcc-4.8.3-20140911/obj-x86_64-redhat-linux/x86_64-redhat-linux/libsanitizer/asan/../../../../libsanitizer/sanitizer_common/sanitizer_common_interceptors.inc:262 </span><span> #2 0x7f59bc0f79e9 in __interceptor_sscanf /usr/src/debug/gcc-4.8.3-20140911/obj-x86_64-redhat-linux/x86_64-redhat-linux/libsanitizer/asan/../../../../libsanitizer/sanitizer_common/sanitizer_common_interceptors.inc:297 </span><span> #3 0x7f59b141e060 in read_metadata.isra.5 /home/wibrown/development/389ds/ds/ldap/servers/slapd/back-ldbm/dblayer.c:5268 </span><span> #4 0x7f59b1426b63 in dblayer_start /home/wibrown/development/389ds/ds/ldap/servers/slapd/back-ldbm/dblayer.c:1587 </span><span> #5 0x7f59b14d698e in ldbm_back_start /home/wibrown/development/389ds/ds/ldap/servers/slapd/back-ldbm/start.c:225 </span><span> #6 0x7f59bbd2dc60 in plugin_call_func /home/wibrown/development/389ds/ds/ldap/servers/slapd/plugin.c:1920 </span><span> #7 0x7f59bbd2e8a7 in plugin_call_one /home/wibrown/development/389ds/ds/ldap/servers/slapd/plugin.c:1870 </span><span> #8 0x7f59bbd2e8a7 in plugin_dependency_startall.isra.10.constprop.13 /home/wibrown/development/389ds/ds/ldap/servers/slapd/plugin.c:1679 </span><span> #9 0x4121c5 in main /home/wibrown/development/389ds/ds/ldap/servers/slapd/main.c:1054 </span><span> #10 0x7f59b8df5af4 in __libc_start_main /usr/src/debug/glibc-2.17-c758a686/csu/libc-start.c:274 </span><span> #11 0x4133b4 in _start (/opt/dirsrv/sbin/ns-slapd+0x4133b4) </span><span>Address 0x7fff49a54ff0 is located at offset 448 in frame &lt;read_metadata.isra.5&gt; of T0&#39;s stack: </span><span> This frame has 7 object(s): </span><span> [32, 33) &#39;delimiter&#39; </span><span> [96, 100) &#39;count&#39; </span><span> [160, 168) &#39;buf&#39; </span><span> [224, 256) &#39;prfinfo&#39; </span><span> [288, 416) &#39;value&#39; </span><span> [448, 960) &#39;attribute&#39; </span><span> [992, 5088) &#39;filename&#39; </span><span>HINT: this may be a false positive if your program uses some custom stack unwind mechanism or swapcontext </span><span> (longjmp and C++ exceptions *are* supported) </span><span>SUMMARY: AddressSanitizer: unknown-crash /usr/src/debug/gcc-4.8.3-20140911/obj-x86_64-redhat-linux/x86_64-redhat-linux/libsanitizer/asan/../../../../libsanitizer/sanitizer_common/sanitizer_common_interceptors_scanf.inc:305 scanf_common </span><span>Shadow bytes around the buggy address: </span><span> 0x1000693429a0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 </span><span> 0x1000693429b0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 </span><span> 0x1000693429c0: 00 00 00 00 00 00 f1 f1 f1 f1 01 f4 f4 f4 f2 f2 </span><span> 0x1000693429d0: f2 f2 04 f4 f4 f4 f2 f2 f2 f2 00 f4 f4 f4 f2 f2 </span><span> 0x1000693429e0: f2 f2 00 00 00 00 f2 f2 f2 f2 00 00 00 00 00 00 </span><span>=&gt;0x1000693429f0: 00 00 00 00 00 00 00 00 00 00 f2 f2 f2 f2[00]00 </span><span> 0x100069342a00: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 </span><span> 0x100069342a10: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 </span><span> 0x100069342a20: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 </span><span> 0x100069342a30: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 f2 f2 </span><span> 0x100069342a40: f2 f2 00 00 00 00 00 00 00 00 00 00 00 00 00 00 </span><span>Shadow byte legend (one shadow byte represents 8 application bytes): </span><span> Addressable: 00 </span><span> Partially addressable: 01 02 03 04 05 06 07 </span><span> Heap left redzone: fa </span><span> Heap righ redzone: fb </span><span> Freed Heap region: fd </span><span> Stack left redzone: f1 </span><span> Stack mid redzone: f2 </span><span> Stack right redzone: f3 </span><span> Stack partial redzone: f4 </span><span> Stack after return: f5 </span><span> Stack use after scope: f8 </span><span> Global redzone: f9 </span><span> Global init order: f6 </span><span> Poisoned by user: f7 </span><span> ASan internal: fe </span><span>==28682== ABORTING </span></code></pre> <p>First lets focus on the stack. Specifically:</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>WRITE of size 513 at 0x7fff49a54ff0 thread T0 </span><span> #0 0x7f59bc0f719e in scanf_common /usr/src/debug/gcc-4.8.3-20140911/obj-x86_64-redhat-linux/x86_64-redhat-linux/libsanitizer/asan/../../../../libsanitizer/sanitizer_common/sanitizer_common_interceptors_scanf.inc:305 </span><span> #1 0x7f59bc0f78b6 in __interceptor_vsscanf /usr/src/debug/gcc-4.8.3-20140911/obj-x86_64-redhat-linux/x86_64-redhat-linux/libsanitizer/asan/../../../../libsanitizer/sanitizer_common/sanitizer_common_interceptors.inc:262 </span><span> #2 0x7f59bc0f79e9 in __interceptor_sscanf /usr/src/debug/gcc-4.8.3-20140911/obj-x86_64-redhat-linux/x86_64-redhat-linux/libsanitizer/asan/../../../../libsanitizer/sanitizer_common/sanitizer_common_interceptors.inc:297 </span><span> #3 0x7f59b141e060 in read_metadata.isra.5 /home/wibrown/development/389ds/ds/ldap/servers/slapd/back-ldbm/dblayer.c:5268 </span><span> #4 0x7f59b1426b63 in dblayer_start /home/wibrown/development/389ds/ds/ldap/servers/slapd/back-ldbm/dblayer.c:1587 </span></code></pre> <p>Now, we can ignore frame 0,1,2. These are all in asan. But, we do own code in frame 3. So lets take a look there as our first port of call.</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>vim ldap/servers/slapd/back-ldbm/dblayer.c +5268 </span><span> </span><span>5262 if (NULL != nextline) { </span><span>5263 *nextline++ = &#39;\0&#39;; </span><span>5264 while (&#39;\n&#39; == *nextline) { </span><span>5265 nextline++; </span><span>5266 } </span><span>5267 } </span><span>5268 sscanf(thisline,&quot;%512[a-z]%c%128s&quot;,attribute,&amp;delimiter,value); /* &lt;---- THIS LINE */ </span><span>5269 if (0 == strcmp(&quot;cachesize&quot;,attribute)) { </span><span>5270 priv-&gt;dblayer_previous_cachesize = strtoul(value, NULL, 10); </span><span>5271 } else if (0 == strcmp(&quot;ncache&quot;,attribute)) { </span><span>5272 number = atoi(value); </span><span>5273 priv-&gt;dblayer_previous_ncache = number; </span><span>5274 } else if (0 == strcmp(&quot;version&quot;,attribute)) { </span></code></pre> <p>So the crash is that we write of size 513 here. Lets look at the function sscanf, to see what's happening.</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>man sscanf </span><span> </span><span>int sscanf(const char *str, const char *format, ...); </span><span>... </span><span>The scanf() family of functions scans input according to format as described below </span><span>... </span></code></pre> <p>So, we know that we are writing something too large here. Lets checkout the size of our values at that point.</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>gdb /opt/dirsrv/sbin/ns-slapd </span><span> </span><span>Reading symbols from /opt/dirsrv/sbin/ns-slapd...done. </span><span>(gdb) set args -d 0 -D /opt/dirsrv/etc/dirsrv/slapd-localhost </span><span>(gdb) break dblayer.c:5268 </span><span>No source file named dblayer.c. </span><span>Make breakpoint pending on future shared library load? (y or [n]) y </span><span>Breakpoint 1 (dblayer.c:5268) pending. </span><span>(gdb) run </span><span>Starting program: /opt/dirsrv/sbin/ns-slapd -d 0 -D /opt/dirsrv/etc/dirsrv/slapd-localhost </span><span>[Thread debugging using libthread_db enabled] </span><span>Using host libthread_db library &quot;/lib64/libthread_db.so.1&quot;. </span><span>Detaching after fork from child process 28690. </span><span>[08/Dec/2015:13:18:08 +1000] - slapd_nss_init: chmod failed for file /opt/dirsrv/etc/dirsrv/slapd-localhost/cert8.db error (2) No such file or directory. </span><span>[08/Dec/2015:13:18:08 +1000] - slapd_nss_init: chmod failed for file /opt/dirsrv/etc/dirsrv/slapd-localhost/key3.db error (2) No such file or directory. </span><span>[08/Dec/2015:13:18:08 +1000] - slapd_nss_init: chmod failed for file /opt/dirsrv/etc/dirsrv/slapd-localhost/secmod.db error (2) No such file or directory. </span><span>[08/Dec/2015:13:18:08 +1000] - 389-Directory/1.3.5 B2015.342.252 starting up </span><span> </span><span>Breakpoint 1, read_metadata (li=0x6028000121c0) at /home/wibrown/development/389ds/ds/ldap/servers/slapd/back-ldbm/dblayer.c:5268 </span><span>5268 sscanf(thisline,&quot;%512[a-z]%c%128s&quot;,attribute,&amp;delimiter,value); </span><span>Missing separate debuginfos, use: debuginfo-install sqlite-3.7.17-6.el7_1.1.x86_64 </span></code></pre> <p>If you are missing more debuginfo, install them, and re-run.</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>(gdb) set print repeats 20 </span><span>(gdb) print thisline </span><span>$6 = 0x600c0015e900 &quot;cachesize:10000000\nncache:0\nversion:5\nlocks:10000\n&quot; </span><span>(gdb) print attribute </span><span>$7 = &quot;\200\275\377\377\377\177\000\000p\275\377\377\377\177\000\000\301\066\031\020\000\000\000\000\243|\023\352\377\177\000\000\377\377\377\377\000\000\000\000\000\253bu\256\066\357oPBS\362\377\177\000\000p\277\377\377\377\177\000\000\300\317\377\377\377\177\000\000\320\356\a\000\b`\000\000\060\277\377\377\377\177\000\000\003\000\000\000\000\000\000\000\346w\377\177\000\020\000\000\262AT\362\377\177\000\000\340-T\362\377\177\000\000p\277\377\377\377\177\000\000\247\277\377\377\377\177\000\000\000\020\000\000\377\177\000\000*\021\346\364&#39;\000\200&lt;\240\300L\352\377\177\000\000\000\000\000\000\000\000\000\000\000\253bu\256\066\357o\003\000\000\000\000\000\000\000\210\275U\362\377\177\000\000i\000\020\000\000\000\000\000&quot;... </span><span>(gdb) print &amp;delimiter </span><span>$8 = 0x7fffffffbbb0 &quot;*\021\346\364\377\177&quot; </span><span>(gdb) print value </span><span>$9 = &quot;A\000\000\000\000\000\000\000\070\276\377\377\377\177\000\000\020\276\377\377\377\177\000\000\001\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\330\000\001\000F`\000\000\200\375\000\000F`\000\000\257O\336\367\377\177\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\001\000\000\000\377\177\000\000\000\000\000\000\000\000\000\000\001\000\000\000\000\000\000\000\200\375\000\000F`\000\000\306c%\352\377\177\000\000\236\061T\362\377\177\000&quot; </span></code></pre> <p>Some of these are some chunky values! Okay, lets try and see which one is a bit too big.</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>(gdb) print sizeof(attribute) </span><span>$10 = 512 </span><span>(gdb) print sizeof(&amp;delimiter) </span><span>$11 = 8 </span><span>(gdb) print sizeof(value) </span><span>$12 = 128 </span></code></pre> <p>So, if our write is size 513, the closest is probably the attribute variable. But it's only size 512? How is this causing an issue?</p> <p>Well, if we look at the sscanf man page again for the substitution that attribute will land in (%512[a-z]) we see:</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>Matches a nonempty sequence of characters from the specified set of accepted characters </span><span>... </span><span>must be enough room for all the characters in the string, plus a terminating null byte. </span></code></pre> <p>So, we have space for 512 chars, which is the size of the attribute block, but we don't have space for the null byte! So lets add it in:</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>5194 char attribute[513]; </span></code></pre> <p>If we keep looking at the man page we see another error too for %128s</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>...next pointer must be a pointer to character array that is long enough to hold the input sequence and the terminating null byte (&#39;\0&#39;), which is added automatically. </span></code></pre> <p>So lets preemptively fix that too.</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>5195 char value[129], delimiter; </span></code></pre> <p>Now rebuild</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>make -C ~/build/ds </span><span>sudo make -C ~/build/ds install </span></code></pre> <p>Lets run slapd and see if it fixed it:</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>sudo -s </span><span>export ASAN_SYMBOLIZER_PATH=/usr/bin/llvm-symbolizer </span><span>export ASAN_OPTIONS=symbolize=1 </span><span>/opt/dirsrv/sbin/ns-slapd -d 0 -D /opt/dirsrv/etc/dirsrv/slapd-localhost </span><span> </span><span>I0&gt; /opt/dirsrv/sbin/ns-slapd -d 0 -D /opt/dirsrv/etc/dirsrv/slapd-localhost </span><span>[08/Dec/2015:13:47:20 +1000] - slapd_nss_init: chmod failed for file /opt/dirsrv/etc/dirsrv/slapd-localhost/cert8.db error (2) No such file or directory. </span><span>[08/Dec/2015:13:47:20 +1000] - slapd_nss_init: chmod failed for file /opt/dirsrv/etc/dirsrv/slapd-localhost/key3.db error (2) No such file or directory. </span><span>[08/Dec/2015:13:47:20 +1000] - slapd_nss_init: chmod failed for file /opt/dirsrv/etc/dirsrv/slapd-localhost/secmod.db error (2) No such file or directory. </span><span>[08/Dec/2015:13:47:20 +1000] - 389-Directory/1.3.5 B2015.342.344 starting up </span><span>[08/Dec/2015:13:47:27 +1000] - slapd started. Listening on All Interfaces port 389 for LDAP requests </span></code></pre> <p>Format this into a patch with git:</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>git commit -a </span><span>git format-patch HEAD~1 </span></code></pre> <p>My patch looks like this</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>From eab0f0e9fc24c1915d2767a87a8f089f6d820955 Mon Sep 17 00:00:00 2001 </span><span>From: William Brown &lt;firstyear at redhat.com&gt; </span><span>Date: Tue, 8 Dec 2015 13:52:29 +1000 </span><span>Subject: [PATCH] Ticket 48372 - ASAN invalid write in dblayer.c </span><span> </span><span>Bug Description: During server start up we attempt to write 513 bytes to a </span><span>buffer that is only 512 bytes long. </span><span> </span><span>Fix Description: Increase the size of the buffer that sscanf writes into. </span><span> </span><span>https://fedorahosted.org/389/ticket/48372 </span><span> </span><span>Author: wibrown </span><span> </span><span>Review by: ??? </span><span>--- </span><span> ldap/servers/slapd/back-ldbm/dblayer.c | 4 ++-- </span><span> 1 file changed, 2 insertions(+), 2 deletions(-) </span><span> </span><span>diff --git a/ldap/servers/slapd/back-ldbm/dblayer.c b/ldap/servers/slapd/back-ldbm/dblayer.c </span><span>index 33506f4..9168c8c 100644 </span><span>--- a/ldap/servers/slapd/back-ldbm/dblayer.c </span><span>+++ b/ldap/servers/slapd/back-ldbm/dblayer.c </span><span>@@ -5191,8 +5191,8 @@ static int read_metadata(struct ldbminfo *li) </span><span> PRFileInfo64 prfinfo; </span><span> int return_value = 0; </span><span> PRInt32 byte_count = 0; </span><span>- char attribute[512]; </span><span>- char value[128], delimiter; </span><span>+ char attribute[513]; </span><span>+ char value[129], delimiter; </span><span> int number = 0; </span><span> dblayer_private *priv = (dblayer_private *)li-&gt;li_dblayer_private; </span><span> </span><span>-- </span><span>2.5.0 </span></code></pre> <p>One more bug fixed! Lets get it commited. If you don't have a FAS account, please email the git format-patch output to <a href="mailto:389-devel@lists.fedoraproject.org">389-devel@lists.fedoraproject.org</a> else, raise a ticket on <a href="https://fedorahosted.org/389">https://fedorahosted.org/389</a></p> <p>43</p> ns-slapd access log notes field Fri, 04 Dec 2015 00:00:00 +0000 Unknown https://fy.blackhats.net.au/blog/2015-12-04-ns-slapd-access-log-notes-field/ https://fy.blackhats.net.au/blog/2015-12-04-ns-slapd-access-log-notes-field/ <h1 id="ns-slapd-access-log-notes-field">ns-slapd access log notes field</h1> <p>It would appear we don't have any documentation for the tricky little notes field in ns-slapd.</p> <p>Sometimes in a search you'll see:</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>[26/Nov/2015:10:22:00 +1000] conn=5 op=1 SRCH base=&quot;&quot; scope=0 notes=&quot;U&quot; filter=&quot;(cn=foo)&quot; attrs=&quot;cn&quot; </span></code></pre> <p>See the notes=&quot;U&quot;? Well, it turns out it's the DS trying to help you out.</p> <p>First, the two to look out for are notes=U and notes=A.</p> <p>notes=A is BAD. You never want to get this one. It means that all candidate attributes in the filter are unindexed, so we need to make a full table scan. This can quickly hit the nsslapd-lookthroughlimit.</p> <p>To rectify this, look at the search, and identify the attributes. Look them up in cn=schema:</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>ldapsearch -H ldap://localhost -b &#39;cn=schema&#39; -x &#39;(objectClass=*)&#39; attributeTypes </span></code></pre> <p>And make sure it has an equality syntax:</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>attributeTypes: ( 2.5.4.3 NAME ( &#39;cn&#39; &#39;commonName&#39; ) SUP name EQUALITY caseIg </span><span> noreMatch SUBSTR caseIgnoreSubstringsMatch SYNTAX 1.3.6.1.4.1.1466.115.121.1. </span><span> 15 X-ORIGIN &#39;RFC 4519&#39; X-DEPRECATED &#39;commonName&#39; ) </span></code></pre> <p>If you don't have an equality syntax, DO NOT ADD AN INDEX. Terrible things will happen!</p> <p>notes=U means one of two things. It means that a candidate attribute in the filter is unindexed, but there is still an indexed candidate. Or it means that the search has hit the idlistscanlimit.</p> <p>If you have the query like below, check your nsslapd indexes. cn is probably indexed, but then you need to add the index for sn. Follow the rules as above, and make sure it has an equality syntax. :</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>&quot;(|(cn=foo)(sn=bar))&quot; </span></code></pre> <p>Second, if that's not the issue, and you think you are hitting idlistscanlimit, you can either:</p> <ul> <li>Adjust it globally</li> <li>Adjust it for the single entry</li> </ul> <p>Doing it on the entry, can cause the query to become sometimes more efficient, because you can de-preference certain indexes. There is more to read about here in the &lt;a href=&quot;<a href="http://www.port389.org/docs/389ds/design/fine-grained-id-list-size.html">http://www.port389.org/docs/389ds/design/fine-grained-id-list-size.html</a>&quot;&gt;id scan limit docs&lt;/a&gt;.</p> <p>Remember to test offline, in a production replica!</p> <p>40</p> The hidden log features of ns-slapd Fri, 04 Dec 2015 00:00:00 +0000 Unknown https://fy.blackhats.net.au/blog/2015-12-04-the-hidden-log-features-of-ns-slapd/ https://fy.blackhats.net.au/blog/2015-12-04-the-hidden-log-features-of-ns-slapd/ <h1 id="the-hidden-log-features-of-ns-slapd">The hidden log features of ns-slapd</h1> <p>This week I discovered (Or dug up: ns-slapd is old) that we have two hidden logging features. In fact searching for one of them yields no results, searching the other shows a document that says it's undocumented.</p> <p>This post hopes to rectify that.</p> <p>In ns-slapd, during a normal operation you can see what a connected client is searching in the access log, or what they are changing based on the audit log.</p> <p>If on a configuration for a plugin you need to diagnose these operations you can't do this... At least that's what the documentation tells you.</p> <p>You can enable logging for search operations on a plugin through the value:</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>nsslapd-logAccess: on </span></code></pre> <p>You can enabled logging for mod/modrdn/del/add operations on a plugin through the value:</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>nsslapd-logAudit: on </span></code></pre> <p>This will yield logs such as:</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>time: 20151204143353 </span><span>dn: uid=test1,ou=People,dc=example,dc=com </span><span>result: 0 </span><span>changetype: modify </span><span>delete: memberOf </span><span>- </span><span>replace: modifiersname </span><span>modifiersname: cn=MemberOf Plugin,cn=plugins,cn=config </span><span>- </span><span>replace: modifytimestamp </span><span>modifytimestamp: 20151204043353Z </span><span>- </span><span> </span><span>time: 20151204143353 </span><span>dn: cn=Test Managers,ou=Groups,dc=example,dc=com </span><span>result: 0 </span><span>changetype: modify </span><span>delete: member </span><span>member: uid=test1,ou=People,dc=example,dc=com </span><span>- </span><span>replace: modifiersname </span><span>modifiersname: cn=directory manager </span><span>- </span><span>replace: modifytimestamp </span><span>modifytimestamp: 20151204043353Z </span><span>- </span></code></pre> <p>Finally, a new option has been added that will enable both on all plugins in the server.</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>nsslapd-plugin-logging: on </span></code></pre> <p>All of these configurations are bound by and respect the following settings:</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>nsslapd-accesslog-logging-enabled </span><span>nsslapd-auditlog-logging-enabled </span><span>nsslapd-auditfaillog-logging-enabled </span></code></pre> Where does that attribute belong? Fri, 04 Dec 2015 00:00:00 +0000 Unknown https://fy.blackhats.net.au/blog/2015-12-04-where-does-that-attribute-belong/ https://fy.blackhats.net.au/blog/2015-12-04-where-does-that-attribute-belong/ <h1 id="where-does-that-attribute-belong">Where does that attribute belong?</h1> <p>A lot of the time in ldap, you spend your time scratching your head thinking &quot;Hey, I wish I knew what objectclass I needed for attribute X&quot;.</p> <p>Yes, you can go through the schema, grep out what objectclasses. But it's a bit tedious, and it's also not very accessible.</p> <p>In lib389 I have written a pair of tools to help with this.</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>lib389/clitools/ds_schema_attributetype_list.py </span><span>lib389/clitools/ds_schema_attributetype_query.py </span></code></pre> <p>List does what you expect: It lists the attributes available on a server, but does so neatly compared to ldapsearch -b cn=schema. The output for comparison:</p> <p>ldapsearch -b 'cn=schema' -x '(objectClass=*)' attributeTypes :</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>... </span><span>attributeTypes: ( 1.2.840.113556.1.2.102 NAME &#39;memberOf&#39; DESC &#39;Group that the </span><span> entry belongs to&#39; SYNTAX 1.3.6.1.4.1.1466.115.121.1.12 X-ORIGIN &#39;Netscape Del </span><span> egated Administrator&#39; ) </span><span>... </span></code></pre> <p>python lib389/clitools/ds_schema_attributetype_list.py -i localhost :</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>( 1.2.840.113556.1.2.102 NAME &#39;memberOf&#39; DESC &#39;Group that the entry belongs to&#39; SYNTAX 1.3.6.1.4.1.1466.115.121.1.12 X-ORIGIN &#39;Netscape Delegated Administrator&#39; ) </span></code></pre> <p>The big difference is that it's on one line: Much easier to grep through.</p> <p>The real gem is the query tool.</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>python lib389/clitools/ds_schema_attributetype_query.py -i localhost -a memberOf </span><span>( 1.2.840.113556.1.2.102 NAME &#39;memberOf&#39; DESC &#39;Group that the entry belongs to&#39; SYNTAX 1.3.6.1.4.1.1466.115.121.1.12 X-ORIGIN &#39;Netscape Delegated Administrator&#39; ) </span><span>MUST </span><span>MAY </span><span>( 2.16.840.1.113730.3.2.130 NAME &#39;inetUser&#39; DESC &#39;Auxiliary class which must be present in an entry for delivery of subscriber services&#39; SUP top AUXILIARY MAY ( uid $ inetUserStatus $ inetUserHttpURL $ userPassword $ memberOf ) ) </span><span>( 2.16.840.1.113730.3.2.112 NAME &#39;inetAdmin&#39; DESC &#39;Marker for an administrative group or user&#39; SUP top AUXILIARY MAY ( aci $ memberOf $ adminRole ) ) </span></code></pre> <p>Shows you the attribute, and exactly which objectClasses MAY and MUST host this attribute. Additionally, because we give you the objectClasses too, you can see the implications of which one you want to enable an add to your object.</p> <p>Happy schema querying.</p> <p>&lt;pre&gt;EDIT 2015-12-07 &lt;/pre&gt; Viktor A pointed out that you can do the following:</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>ldapsearch -o ldif-wrap=no -x -b &#39;cn=schema&#39; &#39;(objectClass=*)&#39; attributeTypes </span><span>... </span><span>attributeTypes: ( 2.16.840.1.113730.3.1.612 NAME &#39;generation&#39; DESC &#39;Netscape defined attribute type&#39; SYNTAX 1.3.6.1.4.1.1466.115.121.1.26 X-ORIGIN &#39;Netscape Directory Server&#39; ) </span><span>&lt;/pre&gt; </span><span> </span><span>This will put all the results onto one line rather than wrapping at 80. Additionally, if you find results that are base64ed: </span><span> </span><span>un64ldif () { </span><span> while read l; do </span><span> echo &quot;$l&quot; | grep &#39;^\([^:]\+: \|$\)&#39; || \ </span><span> echo &quot;${l%%:: *}: $(base64 -d &lt;&lt;&lt; &quot;${l#*:: }&quot;)&quot; </span><span> done </span><span> return 0 </span><span>} </span></code></pre> <p>Thanks for the comment! 41</p> Ldap post read control Thu, 26 Nov 2015 00:00:00 +0000 Unknown https://fy.blackhats.net.au/blog/2015-11-26-ldap-post-read-control/ https://fy.blackhats.net.au/blog/2015-11-26-ldap-post-read-control/ <h1 id="ldap-post-read-control">Ldap post read control</h1> <p>This was a bit of a pain to use in python.</p> <p>If we want to modify and entry and immediately check it's entryUSN so that we can track the update status of objects in ldap, we can use the post read control so that after the add/mod/modrdn is complete, we can immediately check the result of usn atomically. This lets us compare entryusn to know if the object has changed or not.</p> <p>To use in python:</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>&gt;&gt;&gt; conn.modify_ext( &#39;cn=Directory Administrators,dc=example,dc=com&#39;, </span><span> ldap.modlist.modifyModlist({}, {&#39;description&#39; : [&#39;oeusoeutlnsoe&#39;] } ), </span><span> [PostReadControl(criticality=True,attrList=[&#39;nsUniqueId&#39;])] </span><span> ) </span><span>6 </span><span>&gt;&gt;&gt; _,_,_,resp_ctrls = conn.result3(6) </span><span>&gt;&gt;&gt; resp_ctrls </span><span>[&lt;ldap.controls.readentry.PostReadControl instance at 0x2389cf8&gt;] </span><span>&gt;&gt;&gt; resp_ctrls[0].dn </span><span>&#39;cn=Directory Administrators,dc=example,dc=com&#39; </span><span>&gt;&gt;&gt; resp_ctrls[0].entry </span><span>{&#39;nsUniqueId&#39;: [&#39;826cc526-8caf11e5-93ba8a51-c5ee9f85&#39;]} </span></code></pre> <p>See also, <a href="http://www.python-ldap.org/doc/html/ldap-controls.html">PostRead</a> and <a href="http://www.python-ldap.org/doc/html/ldap.html">python-ldap</a>.</p> Magic script for post install interface configuration Thu, 26 Nov 2015 00:00:00 +0000 Unknown https://fy.blackhats.net.au/blog/2015-11-26-magic-script-for-post-install-interface-configuration/ https://fy.blackhats.net.au/blog/2015-11-26-magic-script-for-post-install-interface-configuration/ <h1 id="magic-script-for-post-install-interface-configuration">Magic script for post install interface configuration</h1> <p>Generally on a network we can't always trust dhcp or rtadvd to be there for servers.</p> <p>So here is a magic script that will generate an ifcfg based on these parameters when the server first runs. It helps if you register off the mac to a dhcp entry too.</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>DEV=$(ip route | grep ^default | sed &#39;s/^.* dev //;s/ .*$//&#39;|head -1) </span><span>if [ -n &quot;$DEV&quot; ] </span><span>then </span><span> IP_AND_PREFIX_LEN=$(ip -f inet addr show dev $DEV | grep &#39;inet &#39;| head -1 | sed &#39;s/^ *inet *//;s/ .*$//&#39;) </span><span> IP=$(echo ${IP_AND_PREFIX_LEN} | cut -f1 -d&#39;/&#39;) </span><span> MASK=$(ipcalc -m ${IP_AND_PREFIX_LEN} | sed &#39;s/^.*=//&#39;) </span><span> GW=$(ip route | grep default | head -1 | sed &#39;s/^.*via //;s/ .*$//&#39;) </span><span> IP6_PREFIX=$(ip -f inet6 addr show dev $DEV | grep &#39;inet6 &#39;| head -1 | sed &#39;s/^ *inet6 *//;s/ .*$//&#39;) </span><span> IP6=$(echo ${IP6_PREFIX} | cut -f1 -d&#39;/&#39;) </span><span> MASK6=$(echo ${IP6_PREFIX} | cut -f2 -d&#39;/&#39;) </span><span> GW6=$(ip -6 route | grep default | head -1 | sed &#39;s/^.*via //;s/ .*$//&#39;) </span><span> MAC=$(ip link show dev ${DEV} | grep &#39;link/ether &#39;| head -1 | sed &#39;s/^ *link\/ether *//;s/ .*$//&#39;) </span><span> </span><span>cat &gt; /etc/sysconfig/network-scripts/ifcfg-${DEV} &lt;&lt; DEVEOF </span><span># Generated by magic </span><span>DEVICE=${DEV} </span><span>ONBOOT=yes </span><span>NETBOOT=no </span><span>BOOTPROTO=static </span><span>TYPE=Ethernet </span><span>NAME=${DEV} </span><span>DEFROUTE=yes </span><span>IPV4_FAILURE_FATAL=yes </span><span>IPV6_FAILURE_FATAL=yes </span><span>IPV6INIT=yes </span><span>PEERDNS=no </span><span>PEERROUTES=no </span><span>IPV6_AUTOCONF=no </span><span>HWADDR=${MAC} </span><span>IPADDR=${IP} </span><span>GATEWAY=${GW} </span><span>NETMASK=${MASK} </span><span>IPV6ADDR=${IP6} </span><span>#PREFIX=${MASK6} </span><span>IPV6_DEFAULTGW=${GW6} </span><span>DNS1=PUT YOUR DNS SERVER IP HERE </span><span>DNS2=PUT YOUR DNS SERVER IP HERE </span><span>#NM_CONTROLLED=no </span><span> </span><span>DEVEOF </span><span> </span><span> #Done </span><span>fi </span></code></pre> python gssapi with flask and s4u2proxy Thu, 26 Nov 2015 00:00:00 +0000 Unknown https://fy.blackhats.net.au/blog/2015-11-26-python-gssapi-with-flask-and-s4u2proxy/ https://fy.blackhats.net.au/blog/2015-11-26-python-gssapi-with-flask-and-s4u2proxy/ <h1 id="python-gssapi-with-flask-and-s4u2proxy">python gssapi with flask and s4u2proxy</h1> <p><a href="/blog/html/2017/05/23/kerberos_why_the_world_moved_on.html">UPDATE: 2019 I don't recommend using kerberos - read more here.</a></p> <p>I have recently been implementing gssapi negotiate support in a flask application at work. In almost every case I advise that you use mod-auth-gssapi: It's just better.</p> <p>But if you have a use case where you cannot avoid implementing you own, there are some really gotchas in using python-gssapi.</p> <p>Python-gssapi is the updated, newer, better gssapi module for python, essentially obsoleting python-kerberos. It will have python 3 support and is more full featured.</p> <p>However, like everything to do with gssapi, it's fiendishly annoying to use, and lacks a lot in terms of documentation and examples.</p> <p>The hardest parts:</p> <ul> <li>Knowing how to complete the negotiation with the data set in headers by the client</li> <li>Finding that python-gssapi expects you to base64 decode the request</li> <li>Finding how to destroy credentials</li> <li>Getting the delegated credentials into a ccache</li> </ul> <p>Now, a thing to remember is that here, if your kdc support it, you will be using s4u2proxy automatically. If you want to know more, and you are using freeipa, you can look into <a href="http://www.freeipa.org/page/V4/Service_Constraint_Delegation">constrained delegation</a>.</p> <p>Here is how I implemented the negotiate handler in flask.</p> <p>:</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>def _negotiate_start(req): </span><span> # This assumes a realm. You can leave this unset to use the default ream from krb5.conf iirc. </span><span> svc_princ = gssnames.Name(&#39;HTTP/%s@EXAMPLE.COM&#39;% (socket.gethostname())) </span><span> server_creds = gsscreds.Credentials(usage=&#39;accept&#39;, name=svc_princ) </span><span> context = gssctx.SecurityContext(creds=server_creds) </span><span> # Yay! Undocumented gssapi magic. No indication that you need to b64 decode. </span><span> context.step(base64.b64decode(req)) </span><span> deleg_creds = context.delegated_creds </span><span> CCACHE = &#39;MEMORY:ccache_rest389_%s&#39; % deleg_creds.name </span><span> store = {&#39;ccache&#39;: CCACHE} </span><span> deleg_creds.store(store, overwrite=True) </span><span> os.environ[&#39;KRB5CCNAME&#39;] = CCACHE </span><span> # Return the context, so we can free it later. </span><span> return context </span><span> </span><span> </span><span>def _negotiate_end(context): </span><span> </span><span> # tell python-gssapi to free gss_cred_id_t </span><span> deleg_creds = context.delegated_creds </span><span> del(deleg_creds) </span><span> </span><span> </span><span>def _connection(f, *args, **kwargs): </span><span> retval = None </span><span> negotiate = False </span><span> headers = Headers() # Allows a multivalue header response. </span><span> # Request comes from **kwargs </span><span> authorization = request.headers.get(&quot;Authorization&quot;, None) </span><span> try: </span><span> if authorization is not None: </span><span> values = authorization.split() </span><span> if values[0] == &#39;Negotiate&#39;: </span><span> # If this is valid, it sets KRB5CCNAME </span><span> negotiate = _negotiate_start(values[1]) </span><span> # This is set by mod_auth_gssapi if you are using that instead. </span><span> if request.headers.get(&quot;Krb5Ccname&quot;, &#39;(null)&#39;) != &#39;(null)&#39;: </span><span> os.environ[&#39;KRB5CCNAME&#39;] = request.headers.get(&quot;Krb5Ccname&quot;, None) </span><span> if os.environ.get(&#39;KRB5CCNAME&#39;, &#39;&#39;) != &#39;&#39;: </span><span> pass </span><span> # Do something with the krb creds here, db connection etc. </span><span> retval = f(dir_srv_conn, *args, **kwargs) </span><span> else: </span><span> headers.add(&#39;WWW-Authenticate&#39;, &#39;Negotiate&#39;) </span><span> retval = Response(&quot;Unauthorized&quot;, 401, headers) </span><span> finally: </span><span> if negotiate is not False: </span><span> _negotiate_end(negotiate) </span><span> if os.environ.get(&#39;KRB5CCNAME&#39;, None) is not None: </span><span> os.environ[&#39;KRB5CCNAME&#39;] = &#39;&#39; </span><span> return retval </span><span> </span><span> </span><span>def authenticateConnection(f): </span><span> @wraps(f) </span><span> def decorator(*args, **kwargs): </span><span> return _connection(f, *args, **kwargs) </span><span> return decorator </span><span> </span><span> </span><span>@app.route(&#39;/&#39;, methods[&#39;GET&#39;]) </span><span>@authenticateConnection </span><span>def index(): </span><span> pass </span></code></pre> Managing replication conflicts for humans in 389 Wed, 25 Nov 2015 00:00:00 +0000 Unknown https://fy.blackhats.net.au/blog/2015-11-25-managing-replication-conflicts-for-humans-in-389/ https://fy.blackhats.net.au/blog/2015-11-25-managing-replication-conflicts-for-humans-in-389/ <h1 id="managing-replication-conflicts-for-humans-in-389">Managing replication conflicts for humans in 389</h1> <p>I would like to thank side_control at runlevelone dot net for putting me onto this challenge.</p> <p>If we have a replication conflict in 389, we generall have two results. A and B. In the case A is the live object and B is the conflict, and we want to keep A as live object, it's as easy as:</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>dn: idnsname=_kerberos._udp.Default-First-Site-Name._sites.dc._msdcs+nsuniqueid=910d8837-4c3c11e5-83eea63b-366c3f94,idnsname=lab.example.lan.,cn=dns,dc=lab,dc=example,dc=lan </span><span>changetype: delete </span></code></pre> <p>But say we want to swap them over: We want to keep B, but A is live. How do we recover this?</p> <p>I plan to make a tool to do this, because it's a right pain.</p> <p>This is the only way I got it to work, but I suspect there is a shortcut somewhere that doesn't need the blackmagic that is extensibleObject. (If you use extensibleObject in production I will come for your personally.)</p> <p>First, we need to get the object out of being a multivalued rdn object so we can manipulate it easier. We give it a cn to match it's uniqueId.</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>dn: idnsname=_kerberos._udp.dc._msdcs+nsuniqueid=910d8842-4c3c11e5-83eea63b-366c3f94,idnsname=lab.example.lan.,cn=dns,dc=lab,dc=example,dc=lan </span><span>changetype: modify </span><span>add: cn </span><span>cn: 910d8842-4c3c11e5-83eea63b-366c3f94 </span><span>- </span><span>replace: objectClass </span><span>objectClass: extensibleObject </span><span>objectClass: idnsrecord </span><span>objectClass: top </span><span>- </span><span> </span><span>dn: idnsname=_kerberos._udp.dc._msdcs+nsuniqueid=910d8842-4c3c11e5-83eea63b-366c3f94,idnsname=lab.example.lan.,cn=dns,dc=lab,dc=example,dc=lan </span><span>changetype: modrdn </span><span>newrdn: cn=910d8842-4c3c11e5-83eea63b-366c3f94 </span><span>deleteoldrdn: 0 </span><span>newsuperior: </span><span>idnsname=lab.example.lan.,cn=dns,dc=lab,dc=example,dc=lan </span></code></pre> <p>Now, we can get rid of the repl conflict:</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>dn: cn=910d8842-4c3c11e5-83eea63b-366c3f94,idnsname=lab.example.lan.,cn=dns,dc=lab,dc=example,dc=lan </span><span>changetype: modify </span><span>delete: nsds5ReplConflict </span><span>nsds5ReplConflict: </span><span>namingConflictidnsname=_kerberos._udp.dc._msdcs,idnsname=lab.example.lan.,cn=dns,dc=lab,dc=example,dc=lan </span><span>- </span></code></pre> <p>We have &quot;B&quot; ready to go. So lets get A out of the way, and drop B in.</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>dn: idnsname=_kerberos._udp.Default-First-Site-Name._sites.dc._msdcs+nsuniqueid=910d8837-4c3c11e5-83eea63b-366c3f94,idnsname=lab.example.lan.,cn=dns,dc=lab,dc=example,dc=lan </span><span>changetype: delete </span><span> </span><span>dn: cn=910d8842-4c3c11e5-83eea63b-366c3f94,idnsname=lab.example.lan.,cn=dns,dc=lab,dc=example,dc=lan </span><span>changetype: modrdn </span><span>newrdn: idnsName=_kerberos._udp.dc._msdcs </span><span>deleteoldrdn: 0 </span><span>newsuperior: idnsname=lab.example.lan.,cn=dns,dc=lab,dc=example,dc=lan </span></code></pre> <p>Finally, we need to fix the objectClass and get rid of the cn.</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>dn: idnsName=_kerberos._udp.dc._msdcs,idnsname=lab.example.lan.,cn=dns,dc=lab,dc=example,dc=lan </span><span>changetype: modify </span><span>delete: cn </span><span>cn: 910d8842-4c3c11e5-83eea63b-366c3f94 </span><span>- </span><span>replace: objectClass </span><span>objectClass: idnsrecord </span><span>objectClass: top </span><span>- </span></code></pre> <p>I think a tool to do this would be really helpful.</p> Securing RHEL - CentOS - Fedora Sun, 15 Nov 2015 00:00:00 +0000 Unknown https://fy.blackhats.net.au/blog/2015-11-15-securing-rhel-centos-fedora/ https://fy.blackhats.net.au/blog/2015-11-15-securing-rhel-centos-fedora/ <h1 id="securing-rhel-centos-fedora">Securing RHEL - CentOS - Fedora</h1> <p>We've had a prompting to investigate our OS security at my work. As a result, I've been given a pretty open mandate to investigate and deliver some simple changes that help lock down our systems and make measurable changes to security and incident analysis.</p> <p>First, I used some common sense. Second, I did my research. Third, I used tools to help look at things that I would otherwise have missed.</p> <p>The best tool I used was certainly OpenSCAP. Very simple to use, and gives some really basic recommendations that just make sense. Some of it's answers I took with a grain of salt. For example, account lockout modules in pam aren't needed, as we handle this via our directory services. But it can highlight areas you may have missed.</p> <p>To run a scap scan:</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>yum install scap-security-guide openscap openscap-scanner </span><span>FEDORA </span><span>oscap xccdf eval --profile xccdf_org.ssgproject.content_profile_common --results /tmp/`hostname`-ssg-results.xml \ </span><span>--report /tmp/`hostname`-ssg-results.html /usr/share/xml/scap/ssg/content/ssg-fedora-ds.xml </span><span>RHEL / CENTOS </span><span>oscap xccdf eval --profile xccdf_org.ssgproject.content_profile_common --results /tmp/`hostname`-ssg-results.xml \ </span><span>--report /tmp/`hostname`-ssg-results.html /usr/share/xml/scap/ssg/content/ssg-rhel7-ds.xml </span></code></pre> <p>Then view the output in a web browser.</p> <p>Here is what I came up with.</p> <p>-- Partitioning --------------------</p> <p>Sadly, you need to reinstall for these, but worth rolling out for &quot;future builds&quot;. Here is my partition section from ks.conf. Especially important is putting audit on its own partition.</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span># Partition clearing information </span><span>bootloader --location=mbr </span><span>clearpart --initlabel --all </span><span># Disk partitioning information </span><span>part /boot --fstype=ext4 --size=512 --asprimary --fsoptions=x-systemd.automount,nodev,nosuid,defaults </span><span># LVM </span><span>part pv.2 --size=16384 --grow --asprimary </span><span>volgroup vg00 pv.2 </span><span>logvol swap --fstype=swap --size=2048 --name=swap_lv --vgname=vg00 </span><span>logvol / --fstype=xfs --size=512 --name=root_lv --vgname=vg00 --fsoptions=defaults </span><span>logvol /usr --fstype=xfs --size=3072 --name=usr_lv --vgname=vg00 --fsoptions=nodev,defaults </span><span>logvol /home --fstype=&quot;xfs&quot; --size=512 --name=home_lv --vgname=vg00 --fsoptions=nodev,nosuid,defaults </span><span>logvol /var --fstype=xfs --size=3072 --name=var_lv --vgname=vg00 --fsoptions=nodev,nosuid,noexec,defaults </span><span>logvol /var/log --fstype=&quot;xfs&quot; --size=1536 --name=var_log_lv --vgname=vg00 --fsoptions=nodev,nosuid,noexec,defaults </span><span>logvol /var/log/audit --fstype=&quot;xfs&quot; --size=512 --name=var_log_audit_lv --vgname=vg00 --fsoptions=nodev,nosuid,noexec,defaults </span><span>logvol /srv --fstype=&quot;xfs&quot; --size=512 --name=srv_lv --vgname=vg00 --fsoptions=nodev,nosuid,defaults </span><span>logvol /opt --fstype=&quot;xfs&quot; --size=512 --name=opt_lv --vgname=vg00 --fsoptions=nodev,nosuid,defaults </span></code></pre> <p>With /tmp, if you mount this, and run redhat satellite, you need to be careful. Satellite expects to be able to execute out of /tmp, so don't set noexec on that partition!</p> <p>-- SSH keys ----------------</p> <p>It's just good practice to use these. It saves typing in a password to a prompt which helps to limit credential exposure. We are enabling LDAP backed SSH keys now to make this easier in our workplace.</p> <p>-- SELinux ---------------</p> <p>SELinux isn't perfect by any means, but it helps a lot. It can make the work of an attacker more complex, and it can help prevent data leakage via the network. Consider that by default httpd_t cannot make outgoing network connections. This is awesome to prevent data being leaked back to attackers. Well worth the time to setup these policies correctly.</p> <p>If you have to set permissive to make an application work, do it on a per-domain basis with:</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>semanage permissive -a httpd_t </span></code></pre> <p>This way the protections on all other processes are not removed.</p> <p>On some of my systems I even run confined staff users to help prevent mistakes / malware from users. I manage this via FreeIPA.</p> <p>-- Auditing ----------------</p> <p>This allows us to see who / what is altering things on our system. We extended the core auditing rules to include a few extras.</p> <p>/etc/audit/rules.d/audit.rules</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span># This file contains the auditctl rules that are loaded </span><span># whenever the audit daemon is started via the initscripts. </span><span># The rules are simply the parameters that would be passed </span><span># to auditctl. </span><span> </span><span># First rule - delete all </span><span>-D </span><span> </span><span># Increase the buffers to survive stress events. </span><span># Make this bigger for busy systems </span><span>-b 8192 </span><span> </span><span># </span><span># Feel free to add below this line. See auditctl man page </span><span>-w /etc/ -p wa -k etc_modification </span><span> </span><span># Detect login log tampering </span><span>-w /var/log/faillog -p wa -k logins </span><span>-w /var/log/lastlog -p wa -k logins </span><span>-w /var/run/utmp -p wa -k session </span><span>-w /var/log/btmp -p wa -k session </span><span>-w /var/log/wtmp -p wa -k session </span><span> </span><span># audit_time_rules </span><span>## REMOVE STIME ON RHEL </span><span>#-a always,exit -F arch=b32 -S stime -S adjtimex -S settimeofday -S clock_settime -k audit_time_rules </span><span>#-a always,exit -F arch=b64 -S stime -S adjtimex -S settimeofday -S clock_settime -k audit_time_rules </span><span> </span><span># audit_rules_networkconfig_modification </span><span>-a always,exit -F arch=b32 -S sethostname -S setdomainname -k audit_rules_networkconfig_modification </span><span>-a always,exit -F arch=b64 -S sethostname -S setdomainname -k audit_rules_networkconfig_modification </span><span> </span><span># Audit kernel module manipulation </span><span>-a always,exit -F arch=b32 -S init_module -S delete_module -k modules </span><span>-a always,exit -F arch=b64 -S init_module -S delete_module -k modules </span><span> </span><span>################################################################################ </span><span># These are super paranoid rules at this point. Only use if you are willing to take </span><span># a 3% to 10% perf degredation. </span><span> </span><span># Perhaps remove the uid limits on some of these actions? We often get attacked via services, not users. These rules are more for workstations... </span><span> </span><span>#-a always,exit -F arch=b32 -S chmod -S chown -S fchmod -S fchmodat -S fchown -S fchownat -S fremovexattr -S fsetxattr -S lchown -S lremovexattr -S lsetxattr -S removexattr -S setxattr -F auid&gt;=500 -F auid!=4294967295 -k perm_mod </span><span>#-a always,exit -F arch=b32 -S creat -S open -S openat -S open_by_handle_at -S truncate -S ftruncate -F exit=-EACCES -F auid&gt;=500 -F auid!=4294967295 -k access </span><span>#-a always,exit -F arch=b32 -S creat -S open -S openat -S open_by_handle_at -S truncate -S ftruncate -F exit=-EPERM -F auid&gt;=500 -F auid!=4294967295 -k access </span><span>#-a always,exit -F arch=b32 -S rmdir -S unlink -S unlinkat -S rename -S renameat -F auid&gt;=500 -F auid!=4294967295 -k delete </span><span># This rule is more useful on a workstation with automount ... </span><span>#-a always,exit -F arch=b32 -S mount -F auid&gt;=500 -F auid!=4294967295 -k export </span><span> </span><span>#-a always,exit -F arch=b64 -S chmod -S chown -S fchmod -S fchmodat -S fchown -S fchownat -S fremovexattr -S fsetxattr -S lchown -S lremovexattr -S lsetxattr -S removexattr -S setxattr -F auid&gt;=500 -F auid!=4294967295 -k perm_mod </span><span>#-a always,exit -F arch=b64 -S creat -S open -S openat -S open_by_handle_at -S truncate -S ftruncate -F exit=-EACCES -F auid&gt;=500 -F auid!=4294967295 -k access </span><span>#-a always,exit -F arch=b64 -S creat -S open -S openat -S open_by_handle_at -S truncate -S ftruncate -F exit=-EPERM -F auid&gt;=500 -F auid!=4294967295 -k access </span><span>#-a always,exit -F arch=b64 -S rmdir -S unlink -S unlinkat -S rename -S renameat -F auid&gt;=500 -F auid!=4294967295 -k delete </span><span># This rule is more useful on a workstation with automount ... </span><span>#-a always,exit -F arch=b64 -S mount -F auid&gt;=500 -F auid!=4294967295 -k export </span><span> </span><span># This setting means you need a reboot to changed audit rules. </span><span># probably worth doing .... </span><span>#-e 2 </span></code></pre> <p>To handle all the extra events I increased my audit logging sizes</p> <p>/etc/audit/auditd.conf :</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>log_file = /var/log/audit/audit.log </span><span>log_format = RAW </span><span>log_group = root </span><span>priority_boost = 4 </span><span>flush = INCREMENTAL </span><span>freq = 20 </span><span>num_logs = 5 </span><span>disp_qos = lossy </span><span>dispatcher = /sbin/audispd </span><span>name_format = NONE </span><span>max_log_file = 20 </span><span>max_log_file_action = ROTATE </span><span>space_left = 100 </span><span>space_left_action = EMAIL </span><span>action_mail_acct = root </span><span>admin_space_left = 75 </span><span>admin_space_left_action = SUSPEND </span><span>admin_space_left_action = email </span><span>disk_full_action = SUSPEND </span><span>disk_error_action = SUSPEND </span><span>tcp_listen_queue = 5 </span><span>tcp_max_per_addr = 1 </span><span>tcp_client_max_idle = 0 </span><span>enable_krb5 = no </span><span>krb5_principal = auditd </span></code></pre> <p>-- PAM and null passwords ------------------------------</p> <p>Scap noticed that the default config of password-auth-ac contained nullok on some lines. Remove this:</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>BEFORE </span><span>auth sufficient pam_unix.so nullok try_first_pass </span><span>AFTER </span><span>auth sufficient pam_unix.so try_first_pass </span></code></pre> <p>-- Firewall (Backups, SMH, NRPE) -------------------------------------</p> <p>Backup clients (Amanda, netbackup, commvault) tend to have very high privilege, no SELinux, and are security swiss cheese. Similar is true for vendor systems like HP system management homepage, and NRPE (nagios). It's well worth locking these down. Before we had blanket &quot;port open&quot; rules, now these are tighter.</p> <p>In iptables, you should use the &quot;-s&quot; to specify a source range these are allowed to connect from. The smaller the range, the better.</p> <p>In firewalld, you need to use the rich language. Which is a bit more verbose, and finicky than iptables. My rules end up as: :</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>rule family=&quot;ipv4&quot; source address=&quot;10.0.0.0/24&quot; port port=&quot;2381&quot; protocol=&quot;tcp&quot; accept </span></code></pre> <p>For example. Use the firewalld-cmd with the --add-rich-rule, or use ansibles rich_rule options.</p> <p>-- AIDE (HIDS) -------------------</p> <p>Aide is a fantastic and simple file integrity checker. I have an ansible role that I can tack onto the end of all my playbooks to automatically update the AIDE database so that it stays consistent with changes, but will allow us to see out of band changes.</p> <p>The default AIDE config often picks up files that change frequently. I have an aide.conf that still provides function, but without triggering false alarms. I include aide-local.conf so that other teams / staff can add application specific aide monitoring that doesn't conflict with my work.</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span># Example configuration file for AIDE. </span><span> </span><span>@@define DBDIR /var/lib/aide </span><span>@@define LOGDIR /var/log/aide </span><span> </span><span># The location of the database to be read. </span><span>database=file:@@{DBDIR}/aide.db.gz </span><span> </span><span># The location of the database to be written. </span><span>#database_out=sql:host:port:database:login_name:passwd:table </span><span>#database_out=file:aide.db.new </span><span>database_out=file:@@{DBDIR}/aide.db.new.gz </span><span> </span><span># Whether to gzip the output to database </span><span>gzip_dbout=yes </span><span> </span><span># Default. </span><span>verbose=5 </span><span> </span><span>#report_url=file:@@{LOGDIR}/aide.log </span><span>report_url=stdout </span><span>#report_url=stderr </span><span>#NOT IMPLEMENTED report_url=mailto:root@foo.com </span><span>report_url=syslog:LOG_AUTH </span><span> </span><span># These are the default rules. </span><span># </span><span>#p: permissions </span><span>#i: inode: </span><span>#n: number of links </span><span>#u: user </span><span>#g: group </span><span>#s: size </span><span>#b: block count </span><span>#m: mtime </span><span>#a: atime </span><span>#c: ctime </span><span>#S: check for growing size </span><span>#acl: Access Control Lists </span><span>#selinux SELinux security context </span><span>#xattrs: Extended file attributes </span><span>#md5: md5 checksum </span><span>#sha1: sha1 checksum </span><span>#sha256: sha256 checksum </span><span>#sha512: sha512 checksum </span><span>#rmd160: rmd160 checksum </span><span>#tiger: tiger checksum </span><span> </span><span>#haval: haval checksum (MHASH only) </span><span>#gost: gost checksum (MHASH only) </span><span>#crc32: crc32 checksum (MHASH only) </span><span>#whirlpool: whirlpool checksum (MHASH only) </span><span> </span><span>FIPSR = p+i+n+u+g+s+m+c+acl+selinux+xattrs+sha256 </span><span> </span><span># Fips without time because of some database/sqlite issues </span><span>FIPSRMT = p+i+n+u+g+s+acl+selinux+xattrs+sha256 </span><span> </span><span>#R: p+i+n+u+g+s+m+c+acl+selinux+xattrs+md5 </span><span>#L: p+i+n+u+g+acl+selinux+xattrs </span><span>#E: Empty group </span><span>#&gt;: Growing logfile p+u+g+i+n+S+acl+selinux+xattrs </span><span> </span><span># You can create custom rules like this. </span><span># With MHASH... </span><span># ALLXTRAHASHES = sha1+rmd160+sha256+sha512+whirlpool+tiger+haval+gost+crc32 </span><span>ALLXTRAHASHES = sha1+rmd160+sha256+sha512+tiger </span><span># Everything but access time (Ie. all changes) </span><span>EVERYTHING = R+ALLXTRAHASHES </span><span> </span><span># Sane, with multiple hashes </span><span># NORMAL = R+rmd160+sha256+whirlpool </span><span>NORMAL = FIPSR+sha512 </span><span> </span><span># For directories, don&#39;t bother doing hashes </span><span>DIR = p+i+n+u+g+acl+selinux+xattrs </span><span> </span><span># Access control only </span><span>PERMS = p+i+u+g+acl+selinux+xattrs </span><span> </span><span># Logfile are special, in that they often change </span><span>LOG = &gt; </span><span> </span><span># Just do sha256 and sha512 hashes </span><span>LSPP = FIPSR+sha512 </span><span>LSPPMT = FIPSRMT+sha512 </span><span> </span><span># Some files get updated automatically, so the inode/ctime/mtime change </span><span># but we want to know when the data inside them changes </span><span>DATAONLY = p+n+u+g+s+acl+selinux+xattrs+sha256 </span><span> </span><span># Next decide what directories/files you want in the database. </span><span> </span><span>/boot NORMAL </span><span>/bin NORMAL </span><span>/sbin NORMAL </span><span>/usr/bin NORMAL </span><span>/usr/sbin NORMAL </span><span>/lib NORMAL </span><span>/lib64 NORMAL </span><span># These may be too variable </span><span>/opt NORMAL </span><span>/srv NORMAL </span><span># These are too volatile </span><span># We can check USR if we want, but it doesn&#39;t net us much. </span><span>#/usr NORMAL </span><span>!/usr/src </span><span>!/usr/tmp </span><span> </span><span># Check only permissions, inode, user and group for /etc, but </span><span># cover some important files closely. </span><span>/etc PERMS </span><span>!/etc/mtab </span><span># Ignore backup files </span><span>!/etc/.*~ </span><span>/etc/exports NORMAL </span><span>/etc/fstab NORMAL </span><span>/etc/passwd NORMAL </span><span>/etc/group NORMAL </span><span>/etc/gshadow NORMAL </span><span>/etc/shadow NORMAL </span><span>/etc/security/opasswd NORMAL </span><span> </span><span>/etc/hosts.allow NORMAL </span><span>/etc/hosts.deny NORMAL </span><span> </span><span>/etc/sudoers NORMAL </span><span>/etc/sudoers.d NORMAL </span><span>/etc/skel NORMAL </span><span> </span><span>/etc/logrotate.d NORMAL </span><span> </span><span>/etc/resolv.conf DATAONLY </span><span> </span><span>/etc/nscd.conf NORMAL </span><span>/etc/securetty NORMAL </span><span> </span><span># Shell/X starting files </span><span>/etc/profile NORMAL </span><span>/etc/bashrc NORMAL </span><span>/etc/bash_completion.d/ NORMAL </span><span>/etc/login.defs NORMAL </span><span>/etc/zprofile NORMAL </span><span>/etc/zshrc NORMAL </span><span>/etc/zlogin NORMAL </span><span>/etc/zlogout NORMAL </span><span>/etc/profile.d/ NORMAL </span><span>/etc/X11/ NORMAL </span><span> </span><span># Pkg manager </span><span>/etc/yum.conf NORMAL </span><span>/etc/yumex.conf NORMAL </span><span>/etc/yumex.profiles.conf NORMAL </span><span>/etc/yum/ NORMAL </span><span>/etc/yum.repos.d/ NORMAL </span><span> </span><span># Ignore lvm files that change regularly </span><span>!/etc/lvm/archive </span><span>!/etc/lvm/backup </span><span>!/etc/lvm/cache </span><span> </span><span># Don&#39;t scan log by default, because not everything is a &quot;growing log file&quot;. </span><span>!/var/log LOG </span><span>!/var/run/utmp LOG </span><span> </span><span># This gets new/removes-old filenames daily </span><span>!/var/log/sa </span><span># As we are checking it, we&#39;ve truncated yesterdays size to zero. </span><span>!/var/log/aide.log </span><span>!/var/log/journal </span><span> </span><span># LSPP rules... </span><span># AIDE produces an audit record, so this becomes perpetual motion. </span><span># /var/log/audit/ LSPP </span><span>/etc/audit/ LSPP </span><span>/etc/audisp/ LSPP </span><span>/etc/libaudit.conf LSPP </span><span>/usr/sbin/stunnel LSPP </span><span>/var/spool/at LSPP </span><span>/etc/at.allow LSPP </span><span>/etc/at.deny LSPP </span><span>/etc/cron.allow LSPP </span><span>/etc/cron.deny LSPP </span><span>/etc/cron.d/ LSPP </span><span>/etc/cron.daily/ LSPP </span><span>/etc/cron.hourly/ LSPP </span><span>/etc/cron.monthly/ LSPP </span><span>/etc/cron.weekly/ LSPP </span><span>/etc/crontab LSPP </span><span>/var/spool/cron/root LSPP </span><span> </span><span>/etc/login.defs LSPP </span><span>/etc/securetty LSPP </span><span>/var/log/faillog LSPP </span><span>/var/log/lastlog LSPP </span><span> </span><span>/etc/hosts LSPP </span><span>/etc/sysconfig LSPP </span><span> </span><span>/etc/inittab LSPP </span><span>#/etc/grub/ LSPP </span><span>/etc/rc.d LSPP </span><span> </span><span>/etc/ld.so.conf LSPP </span><span> </span><span>/etc/localtime LSPP </span><span> </span><span>/etc/sysctl.conf LSPP </span><span> </span><span>/etc/modprobe.conf LSPP </span><span> </span><span>/etc/pam.d LSPP </span><span>/etc/security LSPP </span><span>/etc/aliases LSPP </span><span>/etc/postfix LSPP </span><span> </span><span>/etc/ssh/sshd_config LSPP </span><span>/etc/ssh/ssh_config LSPP </span><span> </span><span>/etc/stunnel LSPP </span><span> </span><span>/etc/vsftpd.ftpusers LSPP </span><span>/etc/vsftpd LSPP </span><span> </span><span>/etc/issue LSPP </span><span>/etc/issue.net LSPP </span><span> </span><span>/etc/cups LSPP </span><span> </span><span># Check our key stores for tampering. </span><span>/etc/pki LSPPMT </span><span>!/etc/pki/nssdb/ </span><span>/etc/pki/nssdb/cert8.db LSPP </span><span>/etc/pki/nssdb/cert9.db LSPP </span><span>/etc/pki/nssdb/key3.db LSPP </span><span>/etc/pki/nssdb/key4.db LSPP </span><span>/etc/pki/nssdb/pkcs11.txt LSPP </span><span>/etc/pki/nssdb/secmod.db LSPP </span><span> </span><span># Check ldap and auth configurations. </span><span>/etc/openldap LSPP </span><span>/etc/sssd LSPP </span><span> </span><span># Ignore the prelink cache as it changes. </span><span>!/etc/prelink.cache </span><span> </span><span># With AIDE&#39;s default verbosity level of 5, these would give lots of </span><span># warnings upon tree traversal. It might change with future version. </span><span># </span><span>#=/lost\+found DIR </span><span>#=/home DIR </span><span> </span><span># Ditto /var/log/sa reason... </span><span>!/var/log/and-httpd </span><span> </span><span>#/root NORMAL </span><span># Admins dot files constantly change, just check PERMS </span><span>#/root/\..* PERMS </span><span># Check root sensitive files </span><span>/root/.ssh/ NORMAL </span><span>/root/.bash_profile NORMAL </span><span>/root/.bashrc NORMAL </span><span>/root/.cshrc NORMAL </span><span>/root/.tcshrc NORMAL </span><span>/root/.zshrc NORMAL </span><span> </span><span> </span><span>@@include /etc/aide-local.conf </span></code></pre> <p>-- Time ------------</p> <p>Make sure you run an NTP client. I'm a fan of chrony these days, as it's syncs quickly and reliably.</p> <p>-- Collect core dumps and abrt -----------------------------------</p> <p>Install and run kdump and abrtd so you can analyse why something crashed, to determine if it was malicious or not.</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>yum install kexec-tools abrt abrt-cli </span><span>systemctl enable abrtd </span></code></pre> <p>At the same time, you need to alter kdump.conf to dump correctly</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>xfs /dev/os_vg/var_lv </span><span>path /crash </span><span>core_collector makedumpfile -l --message-level 7 -d 23,31 </span><span>default reboot </span></code></pre> <p>Finally, append crashkernel=auto to your grub commandline.</p> <p>-- Sysctl --------------</p> <p>These are an evolved set of sysctls and improvements to our base install that help tune some basic network and other areas to strengthen the network stack and base OS.</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span># Ensure ASLR </span><span>kernel.randomize_va_space = 2 </span><span># limit access to dmesg </span><span>## does this affect ansible facts </span><span>kernel.dmesg_restrict = 1 </span><span> </span><span># Prevent suid binaries core dumping. Helps to prevent memory / data leaks </span><span>fs.suid_dumpable = 0 </span><span> </span><span># https://www.kernel.org/doc/Documentation/networking/ip-sysctl.txt </span><span># Controls IP packet forwarding </span><span>net.ipv4.ip_forward = 0 </span><span> </span><span># Controls source route verification </span><span>net.ipv4.conf.default.rp_filter = 1 </span><span> </span><span># Do not accept source routing </span><span>net.ipv4.conf.default.accept_source_route = 0 </span><span> </span><span># Controls the System Request debugging functionality of the kernel </span><span>kernel.sysrq = 0 </span><span> </span><span># Controls whether core dumps will append the PID to the core filename. </span><span># Useful for debugging multi-threaded applications. </span><span>kernel.core_uses_pid = 1 </span><span># Decrease the time default value for tcp_fin_timeout connection </span><span>net.ipv4.tcp_fin_timeout = 35 </span><span># Decrease the time default value for tcp_keepalive_time connection </span><span>net.ipv4.tcp_keepalive_time = 600 </span><span># Provide more ports and timewait buckets to increase connectivity </span><span>net.ipv4.ip_local_port_range = 8192 61000 </span><span>net.ipv4.tcp_max_tw_buckets = 1000000 </span><span> </span><span>## Network Hardening ## </span><span>net.ipv4.tcp_max_syn_backlog = 4096 </span><span>net.ipv4.conf.all.accept_redirects = 0 </span><span>net.ipv4.conf.all.secure_redirects = 0 </span><span>net.ipv4.conf.default.accept_redirects = 0 </span><span>net.ipv4.conf.default.secure_redirects = 0 </span><span>net.ipv4.icmp_echo_ignore_broadcasts = 1 </span><span>net.ipv4.conf.all.send_redirects = 0 </span><span>net.ipv4.conf.default.send_redirects = 0 </span><span>net.ipv4.icmp_ignore_bogus_error_responses = 1 </span><span> </span><span>net.nf_conntrack_max = 262144 </span></code></pre> KRB5 setup for ldap server testing Thu, 05 Nov 2015 00:00:00 +0000 Unknown https://fy.blackhats.net.au/blog/2015-11-05-krb5-setup-for-ldap-server-testing/ https://fy.blackhats.net.au/blog/2015-11-05-krb5-setup-for-ldap-server-testing/ <h1 id="krb5-setup-for-ldap-server-testing">KRB5 setup for ldap server testing</h1> <p>UPDATE: 2019 this is now automated, but I <a href="/blog/html/2017/05/23/kerberos_why_the_world_moved_on.html">don't recommend using kerberos - read more here.</a></p> <p>This will eventually get automated, but here is a quick krb recipe for testing. Works in docker containers too!</p> <h2 id="krb5-without-ldap-backend">-- krb5 without ldap backend.</h2> <p>Add kerberos.example.com as an entry to /etc/hosts for this local machine. It should be the first entry.</p> <p>Edit /etc/krb5.conf.d/example.com</p> <p>NOTE: This doesn't work, you need to add it to krb5.conf. Why doesn't it work?</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>[realms] </span><span>EXAMPLE.COM = { </span><span> kdc = kerberos.example.com </span><span> admin_server = kerberos.example.com </span><span>} </span><span> </span><span>[domain_realm] </span><span>.example.com = EXAMPLE.COM </span><span>example.com = EXAMPLE.COM </span></code></pre> <p>Edit /var/kerberos/krb5kdc/kdc.conf</p> <p># Note, I think the defalt kdc.conf is good. :</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>[kdcdefaults] </span><span> kdc_ports = 88 </span><span> kdc_tcp_ports = 88 </span><span> </span><span>[realms] </span><span> EXAMPLE.COM = { </span><span> #master_key_type = aes256-cts </span><span> acl_file = /var/kerberos/krb5kdc/kadm5.acl </span><span> dict_file = /usr/share/dict/words </span><span> admin_keytab = /var/kerberos/krb5kdc/kadm5.keytab </span><span> supported_enctypes = aes256-cts:normal aes128-cts:normal des3-hmac-sha1:normal arcfour-hmac:normal camellia256-cts:normal camellia128-cts:normal des-hmac-sha1:normal des-cbc-md5:normal des-cbc-crc:normal </span><span> } </span></code></pre> <p>Now setup the database.</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>/usr/sbin/kdb5_util create -r EXAMPLE.COM -s # Prompts for password. Is there a way to avoid prompt? </span></code></pre> <p>Edit /var/kerberos/krb5kdc/kadm5.acl</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>/usr/sbin/kadmin.local -r EXAMPLE.COM -q listprincs </span></code></pre> <p>Add our LDAP servers</p> <p># There is a way to submit these on the CLI, but I get kadmin.local: Cannot find master key record in database while initializing kadmin.local interface</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>/usr/sbin/kadmin.local -r EXAMPLE.COM </span><span>add_principal -randkey ldap/kerberos.example.com@EXAMPLE.COM </span><span>ktadd -k /opt/dirsrv/etc/dirsrv/slapd-localhost/ldap.keytab ldap/kerberos.example.com </span><span>add_principal -pw password client </span><span>exit </span></code></pre> <p>Start the kdc</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>/usr/sbin/krb5kdc -P /var/run/krb5kdc.pid -r EXAMPLE.COM </span></code></pre> <p>OR</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span># You need to edit /etc/sysconfig/krb5kdc and put -r EXAMPLE.COM into args </span><span>systemctl start krb5kdc </span><span> </span><span>KRB5_TRACE=/tmp/foo kinit client@EXAMPLE.COM </span><span>klist </span><span>Ticket cache: KEYRING:persistent:0:0 </span><span>Default principal: client@EXAMPLE.COM </span><span> </span><span>Valid starting Expires Service principal </span><span>05/11/15 11:35:37 06/11/15 11:35:37 krbtgt/EXAMPLE.COM@EXAMPLE.COM </span></code></pre> <p>Now setup the DS instance.</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span># Note, might be dirsrv in newer installs. </span><span>chown nobody: /opt/dirsrv/etc/dirsrv/slapd-localhost/ldap.keytab </span></code></pre> <p>Add:</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>KRB5_KTNAME=/opt/dirsrv/etc/dirsrv/slapd-localhost/ldap.keytab ; export KRB5_KTNAME </span></code></pre> <p>To /opt/dirsrv/etc/sysconfig/dirsrv-localhost</p> <p>Now restart the DS</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>/opt/dirsrv/etc/rc.d/init.d/dirsrv restart </span></code></pre> <p>Add a client object:</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>uid=client,ou=People,dc=example,dc=com </span><span>objectClass: top </span><span>objectClass: account </span><span>uid: client </span></code></pre> <p>Now check the GSSAPI is working.</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>ldapwhoami -Y GSSAPI -H ldap://kerberos.example.com:389 </span><span>SASL/GSSAPI authentication started </span><span>SASL username: client@EXAMPLE.COM </span><span>SASL SSF: 56 </span><span>SASL data security layer installed. </span><span>dn: uid=client,ou=people,dc=example,dc=com </span></code></pre> <p>All ready to go!</p> <p>I have created some helpers in lib389 that are able to do this now.</p> <p>TODO: How to setup krb5 with ldap backend.</p> <p>create instance:</p> <p>/opt/dirsrv/sbin/setup-ds.pl --silent --debug --file=/home/wibrown/development/389ds/setup.inf</p> <p>Now, add the krb5 schema</p> <p>cd /opt/dirsrv/etc/dirsrv/slapd-localhost/schema ln -s ../../../../../../usr/share/doc/krb5-server-ldap/60kerberos.ldif</p> <p>/opt/dirsrv/etc/rc.d/init.d/dirsrv restart</p> <p>Query the schema:</p> <p>python /home/wibrown/development/389ds/lib389/clitools/ds_schema_attributetype_list.py | grep krb</p> Debugging 389ds tests Mon, 03 Aug 2015 00:00:00 +0000 Unknown https://fy.blackhats.net.au/blog/2015-08-03-debugging-389ds-tests/ https://fy.blackhats.net.au/blog/2015-08-03-debugging-389ds-tests/ <h1 id="debugging-389ds-tests">Debugging 389ds tests</h1> <p>I've always found when writing tests for 389ds that's it's really handy to have the ldif of data and the logs from a unit test available. However, by default, these are stored.</p> <p>I discovered that if you add instance.backupFS() just before your instance.delete() you can keep a full dump of the data and logs from the instance.</p> <p>It can also be useful to call db2ldif before you run the backup so that you have a human readable copy of the data on hand as well.</p> <p>I've found the best pattern is:</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>def tearDown(self): </span><span> if self.instance.exists(): </span><span> self.instance.db2ldif(bename=&#39;userRoot&#39;, suffixes=[DEFAULT_SUFFIX], excludeSuffixes=[], encrypt=False, \ </span><span> repl_data=False, outputfile=&#39;%s/ldif/%s.ldif&#39; % (self.instance.dbdir,INSTANCE_SERVERID )) </span><span> self.instance.clearBackupFS() </span><span> self.instance.backupFS() </span><span> self.instance.delete() </span></code></pre> <p>This puts an ldif dump of the DB into the backup path, we then clear old backups for our test instance (else it won't over-write them), finally, we actually do the backup. You should see:</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>snip ... </span><span>DEBUG:lib389:backupFS add = /var/lib/dirsrv/slapd-effectiverightsds/ldif/effectiverightsds.ldif (/) </span><span>snip ... </span><span>INFO:lib389:backupFS: archive done : /tmp/slapd-effectiverightsds.bck/backup_08032015_092510.tar.gz </span></code></pre> <p>Then you can extract this in /tmp/slapd-instance, and examine your logs and the ldif of what was really in your ldap server at the time.</p> mod selinux on rhel7 Mon, 03 Aug 2015 00:00:00 +0000 Unknown https://fy.blackhats.net.au/blog/2015-08-03-mod-selinux-on-rhel7/ https://fy.blackhats.net.au/blog/2015-08-03-mod-selinux-on-rhel7/ <h1 id="mod-selinux-on-rhel7">mod selinux on rhel7</h1> <p>I have now compiled and testing mod_selinux on el7. I'm trying to get this into EPEL now.</p> <p>To test this once you have done a build.</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>#!/usr/bin/env python </span><span>import cgi </span><span>import cgitb; cgitb.enable() # for troubleshooting </span><span>import selinux </span><span> </span><span>print &quot;Content-type: text/html&quot; </span><span>print </span><span>print &quot;&quot;&quot; </span><span>&lt;html&gt; </span><span>&lt;head&gt;&lt;title&gt;Selinux CGI context&lt;/title&gt;&lt;/head&gt; </span><span>&lt;body&gt; </span><span> &lt;p&gt;Current context is %s&lt;/p&gt; </span><span>&lt;/body&gt; </span><span>&lt;/html&gt; </span><span>&quot;&quot;&quot; % cgi.escape(str(selinux.getcon())) </span></code></pre> <p>Put this cgi into:</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>/var/www/cgi-bin/selinux-c1.cgi </span><span>/var/www/cgi-bin/selinux-c2.cgi </span><span>/var/www/cgi-bin/selinux-c3.cgi </span></code></pre> <p>Now, install and configure httpd.</p> <p>/etc/httpd/conf.d/mod_selinux.conf</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>&lt;VirtualHost *:80&gt; </span><span> DocumentRoot /var/www/html </span><span> </span><span> &lt;LocationMatch /cgi-bin/selinux-c2.cgi&gt; </span><span> selinuxDomainVal *:s0:c2 </span><span> &lt;/LocationMatch&gt; </span><span> &lt;LocationMatch /cgi-bin/selinux-c3.cgi&gt; </span><span> selinuxDomainVal *:s0:c3 </span><span> &lt;/LocationMatch&gt; </span><span> </span><span>&lt;/VirtualHost&gt; </span></code></pre> <p>Now when you load each page you should see different contexts such as: &quot;Current context is [0, 'system_u:system_r:httpd_sys_script_t:s0:c3']&quot;</p> <p>You can easily extend these location-match based contexts onto django project urls etc. Consider you have a file upload. You place that into c1, and then have all other processes in c2. If the url needs to look at the file, then you place that in c1 also.</p> <p>Alternately, you can use this for virtualhost isolation, or even if you feel game, write new policies to allow more complex rules within your application.</p> <p>34</p> Ovirt with ldap authentication source Wed, 15 Jul 2015 00:00:00 +0000 Unknown https://fy.blackhats.net.au/blog/2015-07-15-ovirt-with-ldap-authentication-source/ https://fy.blackhats.net.au/blog/2015-07-15-ovirt-with-ldap-authentication-source/ <h1 id="ovirt-with-ldap-authentication-source">Ovirt with ldap authentication source</h1> <p>I want ovirt to auth to our work's ldap server, but the default engine domain system expects you to have kerberos. There is however a new AAA module that you can use.</p> <p>First, install it</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>yum install ovirt-engine-extension-aaa-ldap </span></code></pre> <p>So we have a look at the package listing to see what could be a good example:</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>rpm -ql ovirt-engine-extension-aaa-ldap </span><span>.... </span><span>/usr/share/ovirt-engine-extension-aaa-ldap/examples/simple/ </span></code></pre> <p>So we copy our example in place:</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>cp -r /usr/share/ovirt-engine-extension-aaa-ldap/examples/simple/* /etc/ovirt-engine/ </span></code></pre> <p>Now we edit the values in /etc/ovirt-engine/aaa/profile1.properties to match our site, then restart the engine service.</p> <p>Finally, we need to login is as our admin user, then go to configure and assign our user a role. This should allow them to login.</p> <p>I'm seeing some issues with group permissions at the moment, but I suspect that is a schema mismatch issue.</p> <p>This was a really valuable resource.</p> <p><a href="https://access.redhat.com/documentation/en-US/Red_Hat_Enterprise_Virtualization/3.5/html/Administration_Guide/sect-Directory_Users.html">access.redhat.com</a>.</p> Spamassasin with postfix Fri, 10 Jul 2015 00:00:00 +0000 Unknown https://fy.blackhats.net.au/blog/2015-07-10-spamassasin-with-postfix/ https://fy.blackhats.net.au/blog/2015-07-10-spamassasin-with-postfix/ <h1 id="spamassasin-with-postfix">Spamassasin with postfix</h1> <p>I run my own email servers for the fun of it, and to learn about the best practices etc. I've learnt a lot about email as a result so the exercise has paid off.</p> <p>For about 2 years, I had no spam at all. But for some reason about 5 months ago, suddenly my email address was found, and spam ensued. I didn't want to spend my life hand filtering out the spam, so enter spamasssasin.</p> <p>My mail server config itself is the subject of a different post. Today is just about integrating in spamassassin with postfix.</p> <p>First, make sure we have all the packages we need. I'm a centos/fedora user, so adjust as needed.</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>yum install postfix spamass-milter spamassassin </span></code></pre> <p>The default spamassassin configuration is good, but I'm always open to ideas on how to improve it.</p> <p>Now we configure postfix to pass mail through the spamassasin milter.</p> <p>postfix/main.cf :</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>smtpd_milters = unix:/run/spamass-milter/postfix/sock </span></code></pre> <p>Now, enable our spamassasin and postfix service</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>systemctl enable spamass-milter </span><span>systemctl enable postfix </span></code></pre> <p>Now when you recieve email from spamers, they should be tagged with [SPAM].</p> <p>I use dovecot sieve filters on my mailbox to sort these emails out into a separate spam folder.</p> <p>One of the best things that I learnt with spamassassin is that it's bayesian filters are very powerful if you train them.</p> <p>So I setup a script to help me train the spamassasin bayesian filters. This relies heavily on you as a user manually moving spam that is &quot;missed&quot; from your inbox into your spam folder. You must move it all else this process doesn't work!</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>cd /var/lib/dovecot/vmail/william </span><span>sa-learn --progress --no-sync --ham {.,.INBOX.archive}/{cur,new} </span><span>sa-learn --progress --no-sync --spam .INBOX.spam/{cur,new} </span><span>sa-learn --progress --sync </span></code></pre> <p>First, we learn &quot;real&quot; messages from our inbox and our inbox archive. Then we learn spam from our spam folders. Finally, we commit the new bayes database.</p> <p>This could be extended to multiple users with:</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>cd /var/lib/dovecot/vmail/ </span><span>sa-learn --progress --no-sync --ham {william,otheruser}/{.,.INBOX.archive}/{cur,new} </span><span>sa-learn --progress --no-sync --spam {william,otheruser}/.INBOX.spam/{cur,new} </span><span>sa-learn --progress --sync </span></code></pre> <p>Of course, this completely relies on that user ALSO classifying their mail correctly!</p> <p>However, all users will benefit from the &quot;learning&quot; of a few dedicated users.</p> <p>Some other golden tips for blocking spam, are to set these in postfix's main.cf. Most spammers will violate some of these rules at some point. I often see many blocked because of the invalid helo rules.</p> <p>Note, I don't do &quot;permit networks&quot; because of the way my load balancer is configured.</p> <p>main.cf :</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>smtpd_delay_reject = yes </span><span>smtpd_helo_required = yes </span><span>smtpd_helo_restrictions = </span><span> reject_non_fqdn_helo_hostname, </span><span> reject_invalid_helo_hostname, </span><span> permit </span><span>smtpd_relay_restrictions = permit_sasl_authenticated reject_unauth_destination reject_non_fqdn_recipient reject_unknown_recipient_domain reject_unknown_sender_domain </span><span>smtpd_sender_restrictions = </span><span> reject_non_fqdn_sender, </span><span> reject_unknown_sender_domain, </span><span> permit </span><span>smtpd_recipient_restrictions = reject_unauth_pipelining reject_non_fqdn_recipient reject_unknown_recipient_domain permit_sasl_authenticated reject_unauth_destination permit </span></code></pre> <p>Happy spam hunting!</p> SSH keys in ldap Fri, 10 Jul 2015 00:00:00 +0000 Unknown https://fy.blackhats.net.au/blog/2015-07-10-ssh-keys-in-ldap/ https://fy.blackhats.net.au/blog/2015-07-10-ssh-keys-in-ldap/ <h1 id="ssh-keys-in-ldap">SSH keys in ldap</h1> <p>At the dawn of time, we all used passwords to access systems. It was good, but having to type your password tens, hundreds of times a day got old. So along comes ssh keys. However, as we have grown the number of systems we have it's hard to put your ssh key on all systems easily. Then let alone the mess of needing to revoke an ssh key if it were compromised.</p> <p>Wouldn't it be easier if we could store one copy of your public key, and make it available to all systems? When you revoke that key in one location, it revokes on all systems?</p> <p>Enter ssh public keys in ldap.</p> <p>I think that FreeIPA is a great project, and they enable this by default. However, we all don't have the luxury of just setting up IPA. We have existing systems to maintain, in my case, 389ds.</p> <p>So I had to work out how to setup this system myself.</p> <p>First, you need to setup the LDAP server parts. I applied this ldif:</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>dn: cn=schema </span><span>changetype: modify </span><span>add: attributetypes </span><span>attributetypes: ( 1.3.6.1.4.1.24552.500.1.1.1.13 NAME &#39;sshPublicKey&#39; DESC &#39;MANDATORY: OpenSSH Public key&#39; EQUALITY octetStringMatch SYNTAX 1.3.6.1.4.1.1466.115.121.1.40 ) </span><span>- </span><span>add: objectclasses </span><span>objectClasses: ( 1.3.6.1.4.1.24552.500.1.1.2.0 NAME &#39;ldapPublicKey&#39; SUP top AUXILIARY DESC &#39;MANDATORY: OpenSSH LPK objectclass&#39; MUST ( uid ) MAY ( sshPublicKey ) ) </span><span>- </span><span> </span><span>dn: cn=sshpublickey,cn=default indexes,cn=config,cn=ldbm database,cn=plugins,cn=config </span><span>changetype: add </span><span>cn: sshpublickey </span><span>nsIndexType: eq </span><span>nsIndexType: pres </span><span>nsSystemIndex: false </span><span>objectClass: top </span><span>objectClass: nsIndex </span><span> </span><span>dn: cn=sshpublickey_self_manage,ou=groups,dc=example,dc=com </span><span>changetype: add </span><span>objectClass: top </span><span>objectClass: groupofuniquenames </span><span>cn: sshpublickey_self_manage </span><span>description: Members of this group gain the ability to edit their own sshPublicKey field </span><span> </span><span>dn: dc=example,dc=com </span><span>changetype: modify </span><span>add: aci </span><span>aci: (targetattr = &quot;sshPublicKey&quot;) (version 3.0; acl &quot;Allow members of sshpublickey_self_manage to edit their keys&quot;; allow(write) (groupdn = &quot;ldap:///cn=sshpublickey_self_manage,ou=groups,dc=example,dc=com&quot; and userdn=&quot;ldap:///self&quot; ); ) </span><span>- </span></code></pre> <p>For the keen eyed, this is the schema from openssd-ldap but with the objectClass altered to MAY instead of MUST take sshPublicKey. This allows me to add the objectClass to our staff accounts, without needing to set a key for them.</p> <p>Members of the group in question can now self edit their ssh key. It will look like :</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>dn: uid=william,ou=People,dc=example,dc=com </span><span>sshPublicKey: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDI/xgEMzqNwkXMIjjdDO2+xfru </span><span> HK248uIKZ2CHFGIM+BlEhBjqvLpbrFZDYVsme8997p98ENPHGItFch87GCbRhWrpDHINQAnMQkLvD </span><span> eykE1CpYpMWaeyygRZwCUaFfYJD3OgxVoqcUpAc8ZvtGQmHpo++RD5WPNedXOeq/vZzEPbp96ndOn </span><span> b3WejCxl a1176360@strawberry </span></code></pre> <p>Now we configure SSSD.</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>[domain/foo] </span><span>ldap_account_expire_policy = rhds </span><span>ldap_access_order = filter, expire </span><span>ldap_user_ssh_public_key = sshPublicKey </span><span> </span><span>[sssd] </span><span>... </span><span>services = nss, pam, ssh </span></code></pre> <p>The expire policy is extremely important. In 389ds we set nsAccountLock to true to lock out an account. Normally this would cause the password auth to fail, effectively denying access to servers.</p> <p>However, with ssh keys, this process bypasses the password authentication mechanism. So a valid ssh key could still access a server even if the account lock was set.</p> <p>So we setup this policy, to make sure that the account is locked out from servers even if ssh key authentication is used.</p> <p>This configuration can be tested by running:</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>sss_ssh_authorizedkeys william </span></code></pre> <p>You should see a public key.</p> <p>Finally, we configure sshd to check for these keys</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>AuthorizedKeysCommand /usr/bin/sss_ssh_authorizedkeys </span><span>AuthorizedKeysCommandUser root </span></code></pre> <p>Now you should be able to ssh into your systems.</p> <p>It's a really simple setup to achieve this, and can have some really great outcomes in the business.</p> FreeIPA: Giving permissions to service accounts. Mon, 06 Jul 2015 00:00:00 +0000 Unknown https://fy.blackhats.net.au/blog/2015-07-06-freeipa-giving-permissions-to-service-accounts/ https://fy.blackhats.net.au/blog/2015-07-06-freeipa-giving-permissions-to-service-accounts/ <h1 id="freeipa-giving-permissions-to-service-accounts">FreeIPA: Giving permissions to service accounts.</h1> <p><a href="/blog/html/2019/07/10/i_no_longer_recommend_freeipa.html">I no longer recommend using FreeIPA - Read more here!</a></p> <p>I was setting up FreeRADIUS to work with MSCHAPv2 with FreeIPA (Oh god you're a horrible human being I hear you say).</p> <p>To do this, you need to do a few things, the main one being allowing a service account a read permission to a normally hidden attribute. However, service accounts don't normally have the ability to be added to permission classes.</p> <p>First, to enable this setup, you need to install freeipa-adtrust and do the initial setup.</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>yum install freeipa-server-trust-ad freeradius </span><span> </span><span>ipa-adtrust-install </span></code></pre> <p>Now change an accounts password, then as cn=Directory Manager look at the account. You should see ipaNTHash on the account now.</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>ldapsearch -H ldap://ipa.example.com -x -D &#39;cn=Directory Manager&#39; -W -LLL -Z &#39;(uid=username)&#39; ipaNTHash </span></code></pre> <p>Now we setup the permission and a role to put the service accounts into.</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>ipa permission-add &#39;ipaNTHash service read&#39; --attrs=ipaNTHash --type=user --right=read </span><span>ipa privilege-add &#39;Radius services&#39; --desc=&#39;Privileges needed to allow radiusd servers to operate&#39; </span><span>ipa privilege-add-permission &#39;Radius services&#39; --permissions=&#39;ipaNTHash service read&#39; </span><span>ipa role-add &#39;Radius server&#39; --desc=&quot;Radius server role&quot; </span><span>ipa role-add-privilege --privileges=&quot;Radius services&quot; &#39;Radius server&#39; </span></code></pre> <p>Next, we add the service account.</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>ipa service-add &#39;radius/host.ipa.example.net.au&#39; </span></code></pre> <p>Most services should be able to use the service account with either the keytab for client authentication, or for at least the service to authenticate to ldap. This is how you get the keytab.</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>ipa-getkeytab -p &#39;radius/host.ipa.example.net.au&#39; -s host.ipa.example.net.au -k /root/radiusd.keytab </span><span>kinit -t /root/radiusd.keytab -k radius/host.ipa.example.net.au </span><span>ldapwhoami -Y GSSAPI </span></code></pre> <p>If you plan to use this account with something like radius that only accepts a password, here is how you can set one.</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>dn: krbprincipalname=radius/host.ipa.example.net.au@IPA.EXAMPLE.NET.AU,cn=services,\ </span><span>cn=accounts,dc=ipa,dc=example,dc=net,dc=au </span><span>changetype: modify </span><span>add: objectClass </span><span>objectClass: simpleSecurityObject </span><span>- </span><span>add: userPassword </span><span>userPassword: &lt;The service account password&gt; </span><span> </span><span>ldapmodify -f &lt;path/to/ldif&gt; -D &#39;cn=Directory Manager&#39; -W -H ldap://host.ipa.example.net.au -Z </span><span>ldapwhoami -Z -D &#39;krbprincipalname=radius/host.ipa.example.net.au@IPA.EXAMPLE.NET.AU,\ </span><span>cn=services,cn=accounts,dc=ipa,dc=example,dc=net,dc=au&#39; -W </span></code></pre> <p>For either whoami test you should see a dn like:</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>krbprincipalname=radius/host.ipa.example.net.au@IPA.EXAMPLE.NET.AU,cn=services,cn=accounts,dc=ipa,dc=example,dc=net,dc=au </span></code></pre> <p>Finally, we have to edit the cn=Radius server object and add the service account. This is what the object should look like in the end:</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span># Radius server, roles, accounts, ipa.example.net.au </span><span>dn: cn=Radius server,cn=roles,cn=accounts,dc=ipa,dc=example,dc=net,dc=au </span><span>memberOf: cn=Radius services,cn=privileges,cn=pbac,dc=ipa,dc=example,dc=net, </span><span> dc=au </span><span>memberOf: cn=ipaNTHash service read,cn=permissions,cn=pbac,dc=ipa,dc=example </span><span> ,dc=net,dc=au </span><span>description: Radius server role </span><span>cn: Radius server </span><span>objectClass: groupofnames </span><span>objectClass: nestedgroup </span><span>objectClass: top </span><span>member: krbprincipalname=radius/host.ipa.example.net.au@IPA.EXAMPLE.NET.AU </span><span> ,cn=services,cn=accounts,dc=ipa,dc=example,dc=net,dc=au </span></code></pre> <p>Now you should be able to use the service account to search and show ipaNTHash on objects.</p> <p>If you use this as your identify in raddb/mods-avaliable/ldap, and set control:NT-Password := 'ipaNTHash' in the update section, you should be able to use this as an ldap backend for MSCHAPv2. I will write a more complete blog on the radius half of this setup later.</p> <p>NOTES: Thanks to afayzullin for noting the deprecation of --permission with ipa permission-add. This has been updated to --right as per his suggestion. Additional thanks for pointing out I should include the command to do the directory manager ldapsearch for ipanthash.</p> OpenBSD relayd Sun, 05 Jul 2015 00:00:00 +0000 Unknown https://fy.blackhats.net.au/blog/2015-07-05-openbsd-relayd/ https://fy.blackhats.net.au/blog/2015-07-05-openbsd-relayd/ <h1 id="openbsd-relayd">OpenBSD relayd</h1> <p>I've been using OpenBSD 5.7 as my network router for a while, and I'm always impressed by the tools avaliable.</p> <p>Instead of using direct ipv6 forwarding, or NAT port forwards for services, I've found it a lot easier to use the OpenBSD relayd software to listen on my ingress port, then to relay the traffic in. Additionally, this allows relayd to listen on ipv4 and ipv6 and to rewrite connections to the backend to be purely ipv6.</p> <p>This helps to keep my pf.conf small and clean, and just focussed on security and inter-vlan / vrf traffic.</p> <p>The only changes to pf.conf needed are:</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>anchor &quot;relayd/*&quot; rtable 0 </span></code></pre> <p>The relayd.conf man page is fantastic and detailed. Read through it for help, but my basic config is:</p> <p>:</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>ext_addr=&quot;ipv4&quot; </span><span>ext_addr6=&quot;ipv6&quot; </span><span> </span><span> </span><span>smtp_port=&quot;25&quot; </span><span>smtp_addr=&quot;2001:db8:0::2&quot; </span><span> </span><span>table &lt;smtp&gt; { $smtp_addr } </span><span> </span><span>protocol &quot;tcp_service&quot; { </span><span> tcp { nodelay, socket buffer 65536 } </span><span>} </span><span> </span><span>relay &quot;smtp_ext_forwarder&quot; { </span><span> listen on $ext_addr port $smtp_port </span><span> listen on $ext_addr6 port $smtp_port </span><span> protocol &quot;tcp_service&quot; </span><span> forward to &lt;smtp&gt; port $smtp_port check tcp </span><span>} </span></code></pre> <p>That's it! Additionally, a great benefit is that when the SMTP server goes away, the check tcp will notice the server is down and drop the service. This means that you won't have external network traffic able to poke your boxes when services are down or have been re-iped and someone forgets to disable the load balancer configs.</p> <p>This also gives me lots of visibility into the service and connected hosts:</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>relayctl show sum </span><span>Id Type Name Avlblty Status </span><span>1 relay smtp_ext_forwarder active </span><span>1 table smtp:25 active (1 hosts) </span><span>1 host 2001:db8:0::2 99.97% up </span><span> </span><span>relayctl show sessions </span><span>session 0:53840 X.X.X.X:3769 -&gt; 2001:db8:0::2:25 RUNNING </span><span> age 00:00:01, idle 00:00:01, relay 1, pid 19574 </span></code></pre> <p>So relayd has simplified my router configuration for external services and allows me to see and migrate services internally without fuss of my external configuration.</p> OpenBSD BGP and VRFs Sat, 04 Jul 2015 00:00:00 +0000 Unknown https://fy.blackhats.net.au/blog/2015-07-04-openbsd-bgp-and-vrfs/ https://fy.blackhats.net.au/blog/2015-07-04-openbsd-bgp-and-vrfs/ <h1 id="openbsd-bgp-and-vrfs">OpenBSD BGP and VRFs</h1> <p>VRFs, or in OpenBSD rdomains, are a simple, yet powerful (and sometimes confusing) topic.</p> <p>Simply, when you have a normal router or operating system, you have a single router table. You have network devices attached into this routing table, traffic may be sent between those interfaces, or they may exit via a default route.</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>10.0.1.1/24 </span><span>eth0 --&gt; | | </span><span>10.0.2.1/24| rdomain 0 | --&gt; pppoe0 default route </span><span>eth1 --&gt; | | </span></code></pre> <p>So in this example, traffic from 10.0.1.1 can flow to 10.0.2.1 and vice versa. Traffic that matches neither of these will be sent down the pppoe0 default route.</p> <p>Now, lets show what that looks like with two rdomains:</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>10.0.1.1/24 </span><span>eth0 --&gt; | | </span><span>10.0.2.1/24| rdomain 0 | --&gt; pppoe0 default route </span><span>eth1 --&gt; | | </span><span> ------------- </span><span>10.0.3.1/24| | </span><span>eth2 --&gt; | rdomain 1 | </span><span>10.0.4.1/24| | </span></code></pre> <p>Now, in our example, traffic for interfaces on rdomain 0 will flow to each other as normal. At the same time, traffic between devices in rdomain 1 will flow correctly also. However, no traffic BETWEEN rdomain 0 and rdomain 1 is permitted.</p> <p>This also means you could do:</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>10.0.1.1/24 </span><span>eth0 --&gt; | | </span><span>10.0.2.1/24| rdomain 0 | --&gt; pppoe0 default route </span><span>eth1 --&gt; | | </span><span> ------------- </span><span>10.0.3.1/24| | </span><span>eth2 --&gt; | rdomain 1 | --&gt; pppoe1 different default route </span><span>10.0.4.1/24| | </span><span>eth3 --&gt; | | </span></code></pre> <p>So some networks have one default route, while other networks have a different default route. Guest networks come to mind ;)</p> <p>Or you can do:</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>10.0.1.1/24 </span><span>eth0 --&gt; | | </span><span>10.0.2.1/24| rdomain 0 | </span><span>eth1 --&gt; | | </span><span> ------------- </span><span>10.0.1.1/24| | </span><span>eth2 --&gt; | rdomain 1 | </span><span>10.0.2.1/24| | </span><span>eth3 --&gt; | | </span></code></pre> <p>Note that now our ipv4 ip ranges over lap: However, because the traffic entered an interface on a specific rdomain, the traffic will always stay in that rdomain. Traffic from eth1 to 10.0.1.1/24 will always go to eth0. Never eth2, as that would be crossing rdomains.</p> <p>So rdomains are really powerful for network isolation, security, multiple routers or allowing overlapping ip ranges to be reused.</p> <p>Now, we change to a different tact: BGP. BGP is the border gateway protocol. It allows two routers to distribute routes between them so they are aware of and able to route traffic correctly. For example.</p> <p>:</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>10.0.1.0/24| | IC | | 10.0.2.0/24 </span><span>eth0 --&gt; | router A | &lt;----&gt; | router B | &lt;-- eth1 </span><span> | | | | </span></code></pre> <p>Normally with no assistance router A and B could only see each other via the interconnect IC. 10.0.1.0/24 is a mystery to router B, as is 10.0.2.0/24 from router A. They just aren't connected so they can't route traffic.</p> <p>By enabling BGP from A to B over the interconnect, router A can discover the networks attached to router B and vice versa. With this information, both routers can correctly send traffic destined to the other via the IC as they now know the correct destinations and connections.</p> <p>There are plenty of documents on enabling both of these technologies in isolation: However, I had a hard time finding a document that showed how we do both at the same time. I wanted to build:</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>router A router B </span><span>10.0.1.1/24 10.0.3.1/24 </span><span>eth0 --&gt; | | IC1 | | &lt;-- eth0 </span><span>10.0.2.1/24| rdomain 0 | &lt;----&gt; | rdomain 0 | 10.0.4.1/24 </span><span>eth1 --&gt; | | | | &lt;-- eth1 </span><span>------------- ------------- </span><span>10.0.1.1/24| | IC2 | | 10.0.3.1/24 </span><span>eth2 --&gt; | rdomain 1 | &lt;----&gt; | rdomain 1 | &lt;-- eth2 </span><span>10.0.2.1/24| | | | 10.0.4.1/24 </span><span>eth3 --&gt; | | | | &lt;-- eth3 </span></code></pre> <p>So I wanted the networks of rdomain 0 from router A to be exported to rdomain 0 of router B, and the networks of router A rdomain 1 to be exported into router B rdomain 1.</p> <p>The way this is achieved is with BGP communities. The BGP router makes a single connection from router A to router B. The BGPd process on A, is aware of rdomains and is able to read the complete system rdomain state. Each rdomains route table is exported into a community. For example, you would have AS:0 and AS:1 in my example. On the recieving router, the contents of the community are imported to the assocated rdomain. For example, community AS:0 would be imported to rdomain 0.</p> <p>Now, ignoring all the other configuration of interfaces with rdomains and pf, here is what a basic BGP configuration would look like for router A:</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>AS 64524 </span><span>router-id 172.24.17.1 </span><span>fib-update yes </span><span> </span><span>rdomain 0 { </span><span> rd 64523:0 </span><span> import-target rt 64524:0 </span><span> export-target rt 64524:0 </span><span> </span><span> network inet connected </span><span> network 0.0.0.0/0 </span><span> network inet6 connected </span><span> network ::/0 </span><span> </span><span>} </span><span>rdomain 1 { </span><span> rd 64523:1 </span><span> import-target rt 64524:1 </span><span> export-target rt 64524:1 </span><span> </span><span> network inet connected </span><span> #network 0.0.0.0/0 </span><span> network inet6 connected </span><span> #network ::/0 </span><span> </span><span>} </span><span>group ibgp { </span><span> announce IPv4 unicast </span><span> announce IPv6 unicast </span><span> remote-as 64524 </span><span> neighbor 2001:db8:0:17::2 { </span><span> descr &quot;selena&quot; </span><span> } </span><span> neighbor 172.24.17.2 { </span><span> descr &quot;selena&quot; </span><span> } </span><span>} </span><span> </span><span>deny from any </span><span>allow from any inet prefixlen 8 - 24 </span><span>allow from any inet6 prefixlen 16 - 48 </span><span> </span><span># accept a default route (since the previous rule blocks this) </span><span>#allow from any prefix 0.0.0.0/0 </span><span>#allow from any prefix ::/0 </span><span> </span><span># filter bogus networks according to RFC5735 </span><span>#deny from any prefix 0.0.0.0/8 prefixlen &gt;= 8 # &#39;this&#39; network [RFC1122] </span><span>deny from any prefix 10.0.0.0/8 prefixlen &gt;= 8 # private space [RFC1918] </span><span>deny from any prefix 100.64.0.0/10 prefixlen &gt;= 10 # CGN Shared [RFC6598] </span><span>deny from any prefix 127.0.0.0/8 prefixlen &gt;= 8 # localhost [RFC1122] </span><span>deny from any prefix 169.254.0.0/16 prefixlen &gt;= 16 # link local [RFC3927] </span><span>deny from any prefix 172.16.0.0/12 prefixlen &gt;= 12 # private space [RFC1918] </span><span>deny from any prefix 192.0.2.0/24 prefixlen &gt;= 24 # TEST-NET-1 [RFC5737] </span><span>deny from any prefix 192.168.0.0/16 prefixlen &gt;= 16 # private space [RFC1918] </span><span>deny from any prefix 198.18.0.0/15 prefixlen &gt;= 15 # benchmarking [RFC2544] </span><span>deny from any prefix 198.51.100.0/24 prefixlen &gt;= 24 # TEST-NET-2 [RFC5737] </span><span>deny from any prefix 203.0.113.0/24 prefixlen &gt;= 24 # TEST-NET-3 [RFC5737] </span><span>deny from any prefix 224.0.0.0/4 prefixlen &gt;= 4 # multicast </span><span>deny from any prefix 240.0.0.0/4 prefixlen &gt;= 4 # reserved </span><span> </span><span># filter bogus IPv6 networks according to IANA </span><span>#deny from any prefix ::/8 prefixlen &gt;= 8 </span><span>deny from any prefix 0100::/64 prefixlen &gt;= 64 # Discard-Only [RFC6666] </span><span>deny from any prefix 2001:2::/48 prefixlen &gt;= 48 # BMWG [RFC5180] </span><span>deny from any prefix 2001:10::/28 prefixlen &gt;= 28 # ORCHID [RFC4843] </span><span>deny from any prefix 2001:db8::/32 prefixlen &gt;= 32 # docu range [RFC3849] </span><span>deny from any prefix 3ffe::/16 prefixlen &gt;= 16 # old 6bone </span><span>deny from any prefix fc00::/7 prefixlen &gt;= 7 # unique local unicast </span><span>deny from any prefix fe80::/10 prefixlen &gt;= 10 # link local unicast </span><span>deny from any prefix fec0::/10 prefixlen &gt;= 10 # old site local unicast </span><span>deny from any prefix ff00::/8 prefixlen &gt;= 8 # multicast </span><span> </span><span>allow from any prefix 2001:db8:0::/56 prefixlen &gt;= 64 </span><span># This allow should override the deny 172.16.0.0/12 above. </span><span>allow from any prefix 172.24.0.0/16 prefixlen &gt;= 24 # private space [RFC1918] </span></code></pre> <p>So lets break this down.</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>AS 64524 </span><span>router-id 172.24.17.1 </span><span>fib-update yes </span></code></pre> <p>This configuration snippet defines our AS, our router ID and that we want to update the routing tables (forwarding information base)</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>rdomain 0 { </span><span> rd 64523:0 </span><span> import-target rt 64524:0 </span><span> export-target rt 64524:0 </span><span> </span><span> network inet connected </span><span> network 0.0.0.0/0 </span><span> network inet6 connected </span><span> network ::/0 </span><span> </span><span>} </span></code></pre> <p>This looks similar to the configuration of rdomain 1. We define the community with the rd statement, route distinguisher. We define that we will only be importing routes from the AS:community identifier provided by the other BGP instance. We also define that we are exporting our routes from this rdomain to the specified AS:community.</p> <p>Finally, we define the networks that we will advertise in BGP. We could define these manually, or by stating network inet[6] connected, we automatically will export any interface that exists within this rdomain.</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>group ibgp { </span><span> announce IPv4 unicast </span><span> announce IPv6 unicast </span><span> remote-as 64524 </span><span> neighbor 2001:db8:0:17::2 { </span><span> descr &quot;selena&quot; </span><span> } </span><span> neighbor 172.24.17.2 { </span><span> descr &quot;selena&quot; </span><span> } </span><span>} </span></code></pre> <p>This defines our connection to the other bgp neighbour. A big gotcha here is that BGP4 only exports ipv4 routes over an ipv4 connection, and ipv6 over an ipv6 connection. You must therefore define both ipv4 and ipv6 to export both types of routers to the other router.</p> <p>Finally, the allow / deny statements filter the valid networks that we accept for fib updates. This should always be defined to guarantee that your don't accidentally accept routes that should not be present.</p> <p>Router B has a nearly identical configuration, just change the neighbour definitions over.</p> <p>Happy routing!</p> <p>UPDATE: Thanks to P. Caetano for advice on improving the filter allow/deny section.</p> OpenBSD nat64 Sat, 04 Jul 2015 00:00:00 +0000 Unknown https://fy.blackhats.net.au/blog/2015-07-04-openbsd-nat64/ https://fy.blackhats.net.au/blog/2015-07-04-openbsd-nat64/ <h1 id="openbsd-nat64">OpenBSD nat64</h1> <p>I'm a bit of a fan of ipv6. I would like to move as many of my systems to be ipv6-only but in the current world of dual stack that's just not completely possible. Nat64 helps allow ipv6 hosts connect to the ipv4 internet.</p> <p>Normally you have:</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>ipv4 &lt;-- | ------- |--&gt; ipv4 internet </span><span> | | </span><span>host | gateway | </span><span> | | </span><span>ipv6 &lt;-- | ------- |--&gt; ipv6 internet </span></code></pre> <p>The two protocols are kept seperate, and you need both to connect to the network.</p> <p>In a Nat64 setup, your gate way defines a magic prefix that is routed to it, that is at least a /96 - in other words, it contains a /32 that you can populate with ipv4 addresses. So at home I have a /56:</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>2001:db8::/56 </span></code></pre> <p>Inside of this I have reserved a network:</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>2001:db8:0:64::/64 </span></code></pre> <p>Now, if you change the last 32 bits to an ipv4 address such as:</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>2001:db8:0:64::8.8.8.8 </span></code></pre> <p>Or in &quot;pure&quot; ipv6</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>2001:db8:0:64::808:808 </span></code></pre> <p>This traffic is sent via the default route, and the gateway picks it up. It sees the prefix of 2001:db8:0:64::/96 on the packet, it then removes the last 32 bits and forms an ipv4 address. The data of the packet is encapsulated to ipv4, a session table built and the data sent out. When a response comes back, the session table is consulted, the data is mapped back to the origin ipv6 address and re-encapsulated back to the client.</p> <p>Thus you see:</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>ping6 2001:db8:0:64::8.8.8.8 </span><span>PING 2001:db8:0:64::8.8.8.8(2001:db8:0:64::808:808) 56 data bytes </span><span>64 bytes from 2001:db8:0:64::808:808: icmp_seq=1 ttl=57 time=123 ms </span></code></pre> <p>Or from our previous diagram, you can now construct:</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>ipv4 X | ---- | --&gt; ipv4 internet </span><span> | / | </span><span>host | / | </span><span> | / | </span><span>ipv6 &lt;-- | -------- | --&gt; ipv6 internet </span></code></pre> <p>However, you need a supporting technology. If you were to use this normally, applications don't know how to attach the ipv4 data into the ipv6 prefix. So you need to support this application with DNS64. This allows hostnames that have no AAAA record, to automatically have the A data appended to the Nat64 prefix. For example with out DNS64:</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>dig www.adelaide.edu.au +short A </span><span>129.127.144.141 </span><span> </span><span>dig www.adelaide.edu.au +short AAAA </span></code></pre> <p>Now, if we add DNS64</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>dig www.adelaide.edu.au +short AAAA </span><span>2001:db8:0:64::817f:908d </span></code></pre> <p>Now we can contact this:</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>ping6 www.adelaide.edu.au </span><span>PING 2001:db8:0:64::129.127.144.141(2001:db8:0:64::817f:908d) 56 data bytes </span><span>64 bytes from 2001:db8:0:64::817f:908d: icmp_seq=1 ttl=64 time=130 ms </span></code></pre> <p>So, lets get into the meat of the configuration.</p> <p>First, you need a working Nat64. I'm using openBSD 5.7 on my router, so this is configured purely in pf.conf</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>pass in quick on $interface_int_r0 inet6 from any to 2001:db8:0:64::/96 af-to inet from (egress:0) keep state rtable 0 </span></code></pre> <p>That's all it is! Provided you have working ipv6 already, the addition of this will enable a /96 to now function as your nat64 prefix. Remember, you DO NOT need this /96 on an interface or routed. It exists &quot;in the ether&quot; and pf recognises traffic to the prefix and will automatically convert it to ipv4 traffic exiting on your egress device.</p> <p>Next you need to configure dns64. I like bind9 so here is the config you need:</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>options { </span><span> //.... snip ..... </span><span> dns64 2001:db8:0:64::/96 { </span><span> clients { any; }; </span><span> //Exclude prviate networks we connect to. </span><span> mapped { !172.24.0.0/16; !10.0.0.0/8; any; }; </span><span> suffix ::; </span><span> recursive-only yes; </span><span> }; </span><span> </span><span>} </span></code></pre> <p>Once you restart named, you will have working DNS64, and able to contact the ipv4 internet from ipv4 hosts.</p> <p>The only gotcha I have found is with VPNs. When you VPN from the site or into the site with DNS64/Nat64, often you will find that your DNS will resolve hosts to ipv6 addresses, for example, 2001:db8:0:64::10.0.0.1 and then will be put onto your network egress port, rather than down the VPN: Not ideal at all! So I exclude the ipv4 ranges from my local networks and my work place to avoid these issues.</p> Unit testing LDAP acis for fun and profit Sat, 04 Jul 2015 00:00:00 +0000 Unknown https://fy.blackhats.net.au/blog/2015-07-04-unit-testing-ldap-acis-for-fun-and-profit/ https://fy.blackhats.net.au/blog/2015-07-04-unit-testing-ldap-acis-for-fun-and-profit/ <h1 id="unit-testing-ldap-acis-for-fun-and-profit">Unit testing LDAP acis for fun and profit</h1> <p>My workplace is a reasonably sized consumer of 389ds. We use it for storing pretty much all our most important identity data from allowing people to authenticate, to group and course membership, to email routing and even internet access.</p> <p>As a result, it's a really important service to maintain. We need to treat it as one of the most security sensitive services we run. The definition of security I always come back to is &quot;availability, integrity and confidentiality&quot;. Now, we have a highly available environment, and we use TLS with our data to ensure confidentiality of results and queries. Integrity however, is the main target of this post.</p> <p>LDAP allows objects that exist with in the directory to &quot;bind&quot; (authenticate) and then to manipulate other objects in the directories. A set of ACIs (Access Control Instructions) define what objects can modify other objects and their attributes.</p> <p>ACIs are probably one of the most complex parts in a directory server environment to &quot;get right&quot; (With the exception maybe of VLV).</p> <p>I noticed during a security review of our directories ACIs that took the following pattern.</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>aci: (targetattr !=&quot;cn&quot;)(version 3.0;acl &quot;Self write all but cn&quot;;allow (write)(userdn = &quot;ldap:///self&quot;);) </span><span>aci: (targetattr !=&quot;sn&quot;)(version 3.0;acl &quot;Self write all but sn&quot;;allow (write)(userdn = &quot;ldap:///self&quot;);) </span></code></pre> <p>Now, the rules in question we had were more complex and had more rules, but at their essence looked like this. Seems like an innocuous set of rules. &quot;Allow self write to everything but sn&quot; and &quot;Allow self write to everything but cn&quot;.</p> <p>So at the end we expect to see we can write everything but sn and cn.</p> <p>Lets use the ldap effective permissions capability to check this:</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>/usr/lib64/mozldap/ldapsearch -D &#39;cn=Directory Manager&#39; -w - -b &#39;cn=test,ou=people,dc=example,dc=net,dc=au&#39; \ </span><span>-J &quot;1.3.6.1.4.1.42.2.27.9.5.2:false:dn: cn=test,ou=people,dc=example,dc=net,dc=au&quot; &quot;(objectClass=*)&quot; </span><span> </span><span>version: 1 </span><span>dn: cn=test,ou=People,dc=example,dc=net,dc=au </span><span>objectClass: top </span><span>objectClass: person </span><span>cn: test </span><span>sn: test </span><span>userPassword: </span><span>entryLevelRights: v </span><span>attributeLevelRights: objectClass:rscwo, cn:rscwo, sn:rscwo, userPassword:wo </span></code></pre> <p>What! Why does cn have r[ead] s[search] c[ompare] w[rite] o[bliterate]? That was denied? Same for SN.</p> <p>Well, LDAP treats ACIs as a positive union.</p> <p>So we have:</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>aci 1 = ( objectclass, sn, userpassword) </span><span>aci 2 = ( objectclass, cn, userpassword) </span><span>aci 1 U aci 2 = ( objectclass, sn, cn, userpassword ) </span></code></pre> <p>As a result, our seemingly secure rules, actually were conflicting and causing our directory to be highly insecure!</p> <p>So, easy to change this: First we invert the rules (be explicit in all things) to say targetattr = &quot;userpassword&quot; for example. We shouldn't use != rules because they can even conflict between groups and self.</p> <p>How do we detect these issues though?</p> <p>I wrote a python library called usl (university simple ldap). In this I have a toolset for unit testing our ldap acis.</p> <p>We create a py.test testcase, that states for some set of objects, they should have access to some set of attributes on a second set of objects. IE group admins should have rscwo on all other objects.</p> <p>We can then run these tests and determine if this is or isn't the case. For example, if we wrote two test cases for the above to test that &quot;self has rscwo to all attributes or self except sn which should be rsc&quot; and a second test &quot;self has rscwo to all attributes or self except cn which should be rsc&quot;. Our test cases would have failed, and we would be alerted to these issues.</p> <p>As a result of these tests for our acis I was able to find many more security issues: Such as users who could self modify groups, self modify acis, account lockouts of other users, or even turn themselves into a container object and create children. At the worst one aci actually allowed objects to edit their own aci's which would have allowed them to give themself more access potentially. The largest offender were rules that defined targetattr != rules: Often these were actually allowing access to write attributes that administrators would over look.</p> <p>For example, the rule above allowing all write except cn, would actually allow access to nsAccountLock, nsSizeLimit and other object attributes that don't show up on first inspection. The complete list is below. (Note the addition of the '+' )</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>/usr/lib64/mozldap/ldapsearch -D &#39;cn=Directory Manager&#39; -w - -b &#39;cn=test,ou=people,dc=example,dc=net,dc=au&#39; \ </span><span>-J &quot;1.3.6.1.4.1.42.2.27.9.5.2:false:dn: cn=test,ou=people,dc=example,dc=net,dc=au&quot; &quot;(objectClass=*)&quot; &#39;+&#39; </span><span>version: 1 </span><span>dn: cn=test,ou=People,dc=example,dc=net,dc=au </span><span>entryLevelRights: v </span><span>attributeLevelRights: nsPagedLookThroughLimit:rscwo, passwordGraceUserTime:rsc </span><span> wo, pwdGraceUserTime:rscwo, modifyTimestamp:rscwo, passwordExpWarned:rscwo, </span><span> pwdExpirationWarned:rscwo, internalModifiersName:rscwo, entrydn:rscwo, dITCo </span><span> ntentRules:rscwo, supportedLDAPVersion:rscwo, altServer:rscwo, vendorName:rs </span><span> cwo, aci:rscwo, nsSizeLimit:rscwo, attributeTypes:rscwo, acctPolicySubentry: </span><span> rscwo, nsAccountLock:rscwo, passwordExpirationTime:rscwo, entryid:rscwo, mat </span><span> chingRuleUse:rscwo, nsIDListScanLimit:rscwo, nsSchemaCSN:rscwo, nsRole:rscwo </span><span> , retryCountResetTime:rscwo, tombstoneNumSubordinates:rscwo, supportedFeatur </span><span> es:rscwo, ldapSchemas:rscwo, copiedFrom:rscwo, nsPagedIDListScanLimit:rscwo, </span><span> internalCreatorsName:rscwo, nsUniqueId:rscwo, lastLoginTime:rscwo, creators </span><span> Name:rscwo, passwordRetryCount:rscwo, dncomp:rscwo, vendorVersion:rscwo, nsT </span><span> imeLimit:rscwo, passwordHistory:rscwo, pwdHistory:rscwo, objectClasses:rscwo </span><span> , nscpEntryDN:rscwo, subschemaSubentry:rscwo, hasSubordinates:rscwo, pwdpoli </span><span> cysubentry:rscwo, structuralObjectClass:rscwo, nsPagedSizeLimit:rscwo, nsRol </span><span> eDN:rscwo, createTimestamp:rscwo, accountUnlockTime:rscwo, dITStructureRules </span><span> :rscwo, supportedSASLMechanisms:rscwo, supportedExtension:rscwo, copyingFrom </span><span> :rscwo, nsLookThroughLimit:rscwo, nsds5ReplConflict:rscwo, modifiersName:rsc </span><span> wo, matchingRules:rscwo, governingStructureRule:rscwo, entryusn:rscwo, nssla </span><span> pd-return-default-opattr:rscwo, parentid:rscwo, pwdUpdateTime:rscwo, support </span><span> edControl:rscwo, passwordAllowChangeTime:rscwo, nsBackendSuffix:rscwo, nsIdl </span><span> eTimeout:rscwo, nameForms:rscwo, ldapSyntaxes:rscwo, numSubordinates:rscwo, </span><span> namingContexts:rscwo </span></code></pre> <p>As a result of unit testing our ldap aci's we were able to find many many loop holes in our security, and then we were able to programatically close them all down. Reading the ACI's by hand revealed some issues, but by testing the &quot;expected&quot; aci versus actual behaviour highlighted our edge cases and the complex interactions of LDAP systems.</p> <p>I will clean up and publish the usl tool set in the future to help other people test their own LDAP secuity controls.</p> PPP on OpenBSD with Internode Sat, 30 May 2015 00:00:00 +0000 Unknown https://fy.blackhats.net.au/blog/2015-05-30-ppp-on-openbsd-with-internode/ https://fy.blackhats.net.au/blog/2015-05-30-ppp-on-openbsd-with-internode/ <h1 id="ppp-on-openbsd-with-internode">PPP on OpenBSD with Internode</h1> <p>It's taken me some time to get this to work nicely.</p> <p>First, you'll need to install OpenBSD 56 or 57.</p> <p>Once done, you need to configure your ethernet interface facing your router that you would have setup in pppoe bridge mode.</p> <p>/etc/hostname.vio0 :</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>rdomain 0 </span><span>inet 172.24.17.1 255.255.255.0 NONE </span><span>inet6 2001:db8:17::1 64 </span><span>up </span></code></pre> <p>NOTE: You can ignore the rdomain statement, but I'll cover these is a later blog post.</p> <p>Now you need to configure the pppoe interface.</p> <p>/etc/hostname.pppoe0 :</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>!/bin/sleep 10 </span><span>rdomain 0 </span><span>inet 0.0.0.0 255.255.255.255 NONE pppoedev vio0 authproto chap authname &#39;USERNAME@internode.on.net&#39; authkey &#39;PASSWORD&#39; </span><span>dest 0.0.0.1 </span><span>inet6 eui64 </span><span>up </span><span>!/sbin/route -T 0 add default -ifp pppoe0 0.0.0.1 </span><span>!if [ -f /tmp/dhcp6c ] ; then kill -9 `cat /tmp/dhcp6c`; fi </span><span>!/bin/sleep 5 </span><span>!/usr/local/sbin/dhcp6c -c /etc/dhcp6c.conf -p /tmp/dhcp6c pppoe0 </span><span>!/sbin/route -T 0 add -inet6 default -ifp pppoe0 fe80:: </span></code></pre> <p>That's quite the interface config!</p> <p>You need the first sleep to make sure that vio0 is up before this interface starts. Horrible, but it works.</p> <p>Then you define the interface in the same rdomain, and inet6 eui64 is needed so that you can actually get a dhcp6 lease. Then you bring up the interface. Dest is needed to say that the remote router is the device connected at the other end of the tunnel. We manually add the default route for ipv4, and we start the dhcp6c client (After killing any existing ones). Finally, we add the ipv6 default route down the interface</p> <p>Now, the contents of dhcp6c.conf are below:</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span># tun0/pppoe0 is the PPPoE interface </span><span>interface pppoe0 { </span><span> send ia-pd 0; </span><span>}; </span><span> </span><span># em1 is the modem interface </span><span>interface vio0 { </span><span> information-only; </span><span>}; </span><span> </span><span>id-assoc pd { </span><span># em0 is the interface to the internal network </span><span> prefix-interface vio0 { </span><span> sla-id 23; </span><span> sla-len 8; </span><span> }; </span><span>}; </span></code></pre> <p>These are already configured to make the correct request to internode for a /56. If you only get a /64 you need to tweak sla-len.</p> <p>Most of this is well documented, but the real gotchas are in the hostname.pppoe0 script, especially around the addition of the default route and the addition of dhcp6c.</p> <p>If you are running PF, besides normal NAT setup etc, you'll need this for IPv6 to work:</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>interface_ext_r0=&quot;{pppoe0}&quot; </span><span> </span><span>pass in quick on $interface_ext_r0 inet6 proto udp from fe80::/64 port 547 to fe80::/64 port 546 keep state rtable 0 </span></code></pre> Fabric starting guide Mon, 29 Sep 2014 00:00:00 +0000 Unknown https://fy.blackhats.net.au/blog/2014-09-29-fabric-starting-guide/ https://fy.blackhats.net.au/blog/2014-09-29-fabric-starting-guide/ <h1 id="fabric-starting-guide">Fabric starting guide</h1> <p>After the BB14 conf, I am posting some snippets of fabric tasks. Good luck! Feel free to email me if you have questions.</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span># Fabric snippets post BB14 conf </span><span> </span><span># It should be obvious, but no warranty, expressed or otherwise is provided </span><span># with this code. Use at your own risk. Always read and understand code before </span><span># running it in your environment. Test test test. </span><span> </span><span># William Brown, Geraint Draheim and others: University of Adelaide </span><span># william at adelaide.edu.au </span><span> </span><span>#################################################################### </span><span>#### WARNING: THIS CODE MAY NOT RUN DUE TO LACK OF IMPORT, DEPENDENCIES. </span><span>#### OR OTHER ENVIRONMENTAL CHANGES. THIS IS INTENTIONAL. YOU SHOULD </span><span>#### ADAPT SOME OF THESE TO YOUR OWN ENVIRONMENT AND UNDERSTAND THE CODE </span><span>#################################################################### </span><span> </span><span>## Decorators. These provide wrappers to functions to allow you to enforce state </span><span># checks and warnings to users before they run. Here are some of the most useful </span><span># we have developed. </span><span> </span><span> </span><span>def rnt_verbose(func): </span><span> &quot;&quot;&quot; </span><span> When added to a function, this will add implementation of a global VERBOSE </span><span> flag. The reason it&#39;s not a default, is because not every function is </span><span> converted to use it yet. Just run command:verbose=1 </span><span> &quot;&quot;&quot; </span><span> @wraps(func) </span><span> def inner(*args, **kwargs): </span><span> if kwargs.pop(&quot;verbose&quot;, None) is not None: </span><span> global VERBOSE </span><span> VERBOSE = True </span><span> return func(*args, **kwargs) </span><span> return inner </span><span> </span><span> </span><span>## IMPORTANT NOTE: This decorator MUST be before @parallel </span><span>def rnt_imsure(warning=None): </span><span> &quot;&quot;&quot; </span><span> This is a useful decorator that enforces the user types a message into </span><span> the console before the task will run. This is invaluable for high risk </span><span> tasks, essentially forcing that the user MUST take responsibility for their </span><span> actions. </span><span> &quot;&quot;&quot; </span><span> def decorator(func): </span><span> @wraps(func) </span><span> def inner(*args, **kwargs): </span><span> # pylint: disable=global-statement </span><span> global IMSURE_WARNING </span><span> print(&quot;Forcing task to run in serial&quot;) </span><span> if kwargs.pop(&quot;imsure&quot;, None) is None and IMSURE_WARNING is False: </span><span> if warning is not None: </span><span> print(warning) </span><span> cont = getpass(&#39;If you are sure, type &quot;I know what I am doing.&quot; #&#39;) </span><span> if cont == &#39;I know what I am doing.&#39;: </span><span> IMSURE_WARNING = True </span><span> print(&#39;continuing in 5 seconds ...&#39;) </span><span> time.sleep(5) </span><span> print(&quot;Starting ...&quot;) </span><span> else: </span><span> print(&#39;Exiting : No actions were taken&#39;) </span><span> sys.exit(1) </span><span> return func(*args, **kwargs) </span><span> inner.parallel = False </span><span> inner.serial = True </span><span> return inner </span><span> return decorator </span><span> </span><span>def rnt_untested(func): </span><span> &quot;&quot;&quot; </span><span> This decorator wraps functions that we consider new and untested. It gives </span><span> a large, visual warning to the user that this is the case, and allows </span><span> 5 seconds for them to ctrl+c before continuing. </span><span> &quot;&quot;&quot; </span><span> @wraps(func) </span><span> def inner(*args, **kwargs): </span><span> dragon = &quot;&quot;&quot; </span><span> ____________________________________ </span><span>/ THIS IS AN UNTESTED TASK. THERE \\ </span><span>\\ ARE DRAGONS IN THESE PARTS / </span><span> ------------------------------------ </span><span> \\ / \\ //\\ </span><span> \\ |\\___/| / \\// \\\\ </span><span> /0 0 \\__ / // | \\ \\ </span><span> / / \\/_/ // | \\ \\ </span><span> @_^_@&#39;/ \\/_ // | \\ \\ </span><span> //_^_/ \\/_ // | \\ \\ </span><span> ( //) | \\/// | \\ \\ </span><span> ( / /) _|_ / ) // | \\ _\\ </span><span> ( // /) &#39;/,_ _ _/ ( ; -. | _ _\\.-~ .-~~~^-. </span><span> (( / / )) ,-{ _ `-.|.-~-. .~ `. </span><span> (( // / )) &#39;/\\ / ~-. _ .-~ .-~^-. \\ </span><span> (( /// )) `. { } / \\ \\ </span><span> (( / )) .----~-.\\ \\-&#39; .~ \\ `. \\^-. </span><span> ///.----..&gt; \\ _ -~ `. ^-` ^-_ </span><span> ///-._ _ _ _ _ _ _}^ - - - - ~ ~-- ,.-~ </span><span> /.-~ </span><span>&quot;&quot;&quot; </span><span> # pylint: disable=global-statement </span><span> global DRAGON_WARNING </span><span> if not DRAGON_WARNING: </span><span> print(dragon) </span><span> if kwargs.pop(&quot;dragon&quot;, None) is None: </span><span> time.sleep(5) </span><span> print(&quot;RAWR: Your problem now!!!&quot;) </span><span> DRAGON_WARNING = True </span><span> return func(*args, **kwargs) </span><span> return inner </span><span> </span><span>################################################# </span><span># Atomic locking functions. Provides a full lock, and a read lock. This is so </span><span># that multiple systems, users etc can access servers, but the servers allow </span><span># one and only one action to be occuring. </span><span> </span><span>ATOMIC_LOCK = &quot;/tmp/fsm_atomic.lock&quot; </span><span>ATOMIC_FLOCK = &quot;/tmp/fsm_atomic.flock&quot; </span><span>ATOMIC_LOCK_HOSTS = {} </span><span>LOCAL_HOSTNAME = socket.gethostname() </span><span> </span><span>class AtomicException(Exception): </span><span> pass </span><span> </span><span>@task </span><span>def lock(): </span><span> &quot;&quot;&quot; </span><span> usage: lock </span><span> </span><span> WARNING: DO NOT RUN THIS BY HAND UNLESS YOU KNOW WHAT YOU ARE DOING!!! </span><span> </span><span> Will create the atomic FSM lock. This prevents any other atomic function </span><span> from being able to run. </span><span> &quot;&quot;&quot; </span><span> ### I cannot stress enough, do not change this. </span><span> result = run(&quot;&quot;&quot; </span><span> ( </span><span> flock -n 9 || exit 1 </span><span> touch {lock} </span><span> echo {hostname} &gt; {lock} </span><span> ) 9&gt;{flock} </span><span> &quot;&quot;&quot;.format(lock=ATOMIC_LOCK, flock=ATOMIC_FLOCK, hostname=LOCAL_HOSTNAME) ) </span><span> if result.return_code == 0: </span><span> return True </span><span> return False </span><span> </span><span>@task </span><span>def unlock(): </span><span> &quot;&quot;&quot; </span><span> usage: unlock </span><span> </span><span> WARNING: DO NOT RUN THIS BY HAND UNLESS YOU KNOW WHAT YOU ARE DOING!!! </span><span> </span><span> Will remove the atomic FSM lock. This allows any other atomic function </span><span> from to run. </span><span> </span><span> Only run this if you are sure that it needs to clean out a stale lock. The </span><span> fsm atomic wrapper is VERY GOOD at cleaning up after itself. Only a kill -9 </span><span> to the fabric job will prevent it removing the atomic lock. Check what </span><span> you are doing! Look inside of /tmp/fsm_atomic.lock to see who holds the lock right now! </span><span> &quot;&quot;&quot; </span><span> ### I cannot stress enough, do not change this. </span><span> result = run(&quot;&quot;&quot; </span><span> rm {lock} </span><span> &quot;&quot;&quot;.format(lock=ATOMIC_LOCK)) </span><span> if result == 0: </span><span> return True </span><span> return False </span><span> </span><span>def _lock_check(): </span><span> # pylint: disable=global-statement </span><span> global ATOMIC_LOCK_HOSTS </span><span> atomic_lock = False </span><span> t_owner = False </span><span> if ATOMIC_LOCK_HOSTS.has_key(env.host_string): </span><span> atomic_lock = ATOMIC_LOCK_HOSTS[env.host_string] </span><span> t_owner = True </span><span> if not atomic_lock: </span><span> with hide(&#39;warnings&#39;, &#39;running&#39;): </span><span> result = get(ATOMIC_LOCK, local_path=&quot;/tmp/{host}/{page}&quot;.format( </span><span> page=&quot;fsm_atomic.lock&quot;, host=env.host)) </span><span> if len(result) != 0: </span><span> atomic_lock = True </span><span> return atomic_lock, t_owner </span><span> </span><span>def noop(*args, **kwargs): </span><span> log_local(&#39;No-op for %s&#39; % env.host_string, &#39;NOTICE&#39;) </span><span> </span><span>def rnt_fsm_atomic_r(func): </span><span> &quot;&quot;&quot; </span><span> This decorator wraps functions that relate to the FSM and changing of state. </span><span> It triggers an atomic lock in the FSM to prevent other state changes occuring </span><span> </span><span> Fsm atomic tasks can be nested, only the top level task will manage the lock. </span><span> </span><span> If the lock is already taken, we will NOT allow the task to run. </span><span> &quot;&quot;&quot; </span><span> @wraps(func) </span><span> def inner(*args, **kwargs): </span><span> #If ATOMIC_LOCK_HOSTS then we own the lock, so we can use it. </span><span> # ELSE if we don&#39;t hold ATOMIC_LOCK_HOSTS we should check. </span><span> # Really, only the outer most wrapper should check .... </span><span> with settings(warn_only=True): </span><span> # pylint: disable=global-statement </span><span> global ATOMIC_LOCK_HOSTS </span><span> #We DO care about the thread owner. Consider an exclusive lock above </span><span> # a read lock. If we didn&#39;t check that we own that exclusive lock, </span><span> # we wouldn&#39;t be able to run. </span><span> (atomic_lock, t_owner) = _lock_check() </span><span> allow_run = False </span><span> if not atomic_lock or (atomic_lock and t_owner): </span><span> ### We can run </span><span> allow_run = True </span><span> pass </span><span> elif atomic_lock and not t_owner: </span><span> ### We can&#39;t run. The lock is held, and we don&#39;t own it. </span><span> log_local(&#39;ATOMIC LOCK EXISTS, CANNOT RUN %s&#39; % env.host_string, &#39;NOTICE&#39;) </span><span> elif atomic_lock and t_owner: </span><span> #### THIS SHOULDN&#39;T HAPPEN EVER </span><span> log_local(&#39;ATOMIC LOCK STATE IS INVALID PLEASE CHECK&#39;, &#39;CRITICAL&#39;) </span><span> raise AtomicException(&quot;CRITICAL: ATOIC LOCK STATE IS INVALID PLEASE CHECK, CANNOT RUN %s&quot; % env.host_string) </span><span> elif not atomic_lock and not t_owner: </span><span> ### This means there is no lock, and we don&#39;t own one. We can run. </span><span> pass </span><span> try: </span><span> if allow_run: </span><span> return func(*args, **kwargs) </span><span> else: </span><span> return noop(*args, **kwargs) </span><span> finally: </span><span> pass </span><span> return inner </span><span> </span><span> </span><span>def rnt_fsm_atomic_exc(func): </span><span> &quot;&quot;&quot; </span><span> This decorator wraps functions that relate to the FSM and changing of state. </span><span> It triggers an atomic lock in the FSM to prevent other state changes occuring </span><span> until the task is complete. </span><span> </span><span> Fsm atomic tasks can be nested, only the top level task will manage the lock. </span><span> </span><span> If the lock is already taken, we will NOT allow the task to run. </span><span> </span><span> State is passed to nested calls that also need an atomic lock. </span><span> &quot;&quot;&quot; </span><span> @wraps(func) </span><span> def inner(*args, **kwargs): </span><span> with settings(warn_only=True): </span><span> # pylint: disable=global-statement </span><span> global ATOMIC_LOCK_HOSTS </span><span> (atomic_lock, t_owner) = _lock_check() </span><span> atomic_lock_owner = False </span><span> allow_run = False </span><span> if atomic_lock and t_owner: </span><span> #We have the lock, do nothing. </span><span> pass </span><span> allow_run = True </span><span> elif atomic_lock and not t_owner: </span><span> #Someone else has it, error. </span><span> log_local(&#39;ATOMIC LOCK EXISTS, CANNOT RUN %s&#39; % env.host_string, &#39;IMPORTANT&#39;) </span><span> elif not atomic_lock and t_owner: </span><span> #Error, can&#39;t be in this state. </span><span> log_local(&#39;ATOMIC LOCK STATE IS INVALID PLEASE CHECK&#39;, &#39;CRITICAL&#39;) </span><span> raise AtomicException(&quot;CRITICAL: ATOMIC LOCK STATE IS INVALID PLEASE CHECK, CANNOT RUN %s&quot; % env.host_string) </span><span> elif not atomic_lock and not t_owner: </span><span> # Create the lock. </span><span> if not lock(): </span><span> log_local(&#39;LOCK TAKEN BY ANOTHER PROCESS&#39;, &#39;IMPORTANT&#39;) </span><span> raise AtomicException(&quot;CRITICAL: LOCK TAKEN BY ANOTHER PROCESS&quot;) </span><span> ATOMIC_LOCK_HOSTS[env.host_string] = True </span><span> atomic_lock_owner = True </span><span> allow_run = True </span><span> try: </span><span> if allow_run: </span><span> return func(*args, **kwargs) </span><span> else: </span><span> return noop(*args, **kwargs) </span><span> finally: </span><span> if atomic_lock_owner: </span><span> unlock() </span><span> ATOMIC_LOCK_HOSTS[env.host_string] = False </span><span> return inner </span><span> </span><span>################################################## </span><span># Basic service management. </span><span># </span><span>## This is how you should start. Basic start, stop, and status commands. </span><span> </span><span> </span><span>@task </span><span>@parallel </span><span>def start(): </span><span> &quot;&quot;&quot; </span><span> usage: start </span><span> </span><span> Start the MapleTA database, tomcat and webserver </span><span> &quot;&quot;&quot; </span><span> sudo(&#39;service postgresql start&#39;) </span><span> sudo(&#39;service tomcat6 start&#39;) </span><span> sudo(&#39;service httpd start&#39;) </span><span> </span><span> </span><span>@task </span><span>@parallel </span><span>def stop(): </span><span> &quot;&quot;&quot; </span><span> usage: stop </span><span> </span><span> Stop the MapleTA webserver, tomcat and database </span><span> &quot;&quot;&quot; </span><span> sudo(&#39;service httpd stop&#39;) </span><span> sudo(&#39;service tomcat6 stop&#39;) </span><span> sudo(&#39;service postgresql stop&#39;) </span><span> </span><span> </span><span>@task </span><span>def restart(): </span><span> &quot;&quot;&quot; </span><span> usage: restart </span><span> </span><span> Restart the MapleTA database, tomcat and webserver </span><span> &quot;&quot;&quot; </span><span> stop() </span><span> start() </span><span> </span><span> </span><span>@task </span><span>def status(): </span><span> &quot;&quot;&quot; </span><span> usage: status </span><span> </span><span> Check the status of MapleTA </span><span> &quot;&quot;&quot; </span><span> sudo(&#39;service postgresql status&#39;) </span><span> sudo(&#39;service tomcat6 status&#39;) </span><span> sudo(&#39;service httpd status&#39;) </span><span> </span><span>################################## </span><span># Some blackboard tasks. These rely on some of the above decorators. </span><span># </span><span>### These are well developed, and sometimes rely on code not provided here. This </span><span># in very intentional so that you can read it and get ideas of HOW you should </span><span># build code that works in your environment. </span><span> </span><span># Also shows the usage of decorators and how you should use them to protent tasks </span><span> </span><span>################################### </span><span># Helpers </span><span>################################### </span><span> </span><span>def config_key(key): </span><span> if key.endswith(&#39;=&#39;) is False: </span><span> key += &#39;=&#39; </span><span> return run(&quot;egrep &#39;{key}&#39; {bbconfig} | cut -f2 -d\= &quot;.format(key=key, bbconfig=BB_CONFIG)) </span><span> </span><span># return blackboard database instance </span><span>@task </span><span>@rnt_help </span><span>def get_db_instance(): </span><span> &quot;&quot;&quot; </span><span> usage: get_db_instance </span><span> </span><span> Display the servers current DB instance / SID </span><span> &quot;&quot;&quot; </span><span> x = config_key(&#39;bbconfig.database.server.instancename&#39;) </span><span> return x </span><span> </span><span>@task </span><span>def get_db_credentials(): </span><span> &quot;&quot;&quot; </span><span> usage: get_db_credentials </span><span> </span><span> This will retrieve the DB username and password from the BB server, and </span><span> return them as a dict {hostname:X, sid:X, username:X, password:X} </span><span> &quot;&quot;&quot; </span><span> creds = {&#39;hostname&#39; : None, </span><span> &#39;sid&#39; : None, </span><span> &#39;username&#39; : None, </span><span> &#39;password&#39; : None} </span><span> with hide(&#39;everything&#39;): </span><span> creds[&#39;hostname&#39;] = config_key(&#39;bbconfig.database.server.fullhostname&#39;) </span><span> #TODO: Remove this sid appending line </span><span> creds[&#39;sid&#39;] = config_key(&#39;bbconfig.database.server.instancename&#39;) + &#39;.blackboard.inc&#39; </span><span> creds[&#39;username&#39;] = config_key(&#39;antargs.default.vi.db.name&#39;) </span><span> creds[&#39;password&#39;] = config_key(&#39;antargs.default.vi.db.password&#39;) </span><span> return creds </span><span> </span><span> </span><span>@task </span><span>@parallel </span><span>@rnt_fsm_atomic_exc </span><span>def force_stop(): </span><span> &quot;&quot;&quot; </span><span> usage: force_stop -&gt; atomic </span><span> </span><span> Stop blackboard services on hosts in PARALLEL. This WILL bring down all </span><span> hosts FAST. This does NOT gracefully remove from the pool. This DOES NOT </span><span> check the sis integration queue. </span><span> &quot;&quot;&quot; </span><span> log_blackboard(&quot;Stopping BB&quot;, level=&#39;NOTICE&#39;) </span><span> if test_processes(quit=False) is True: </span><span> sudo(&#39;/data/blackboard/bbctl stop&#39;) </span><span> time.sleep(30) </span><span> cleanup_processes() </span><span> test_processes() </span><span> log_blackboard(&quot;Stopped&quot;, level=&#39;SUCCESS&#39;) </span><span> </span><span>@task </span><span>@serial </span><span>@rnt_fsm_atomic_exc </span><span>def force_restart(): </span><span> &quot;&quot;&quot; </span><span> usage: restart -&gt; atomic </span><span> </span><span> Restart blackboard systems in SERIAL. This is a dumb rolling restart. This </span><span> DOES NOT remove from the pool and DOES NOT check the SIS queue </span><span> &quot;&quot;&quot; </span><span> log_blackboard(&quot;Trying to force restart blackboard&quot;, level=&#39;NOTICE&#39;) </span><span> force_stop() </span><span> time.sleep(60) </span><span> start() </span><span> log_blackboard(&quot;force restart complete&quot;, level=&#39;SUCCESS&#39;) </span><span> </span><span> </span><span>@task </span><span>@rnt_imsure() </span><span>def pushconfigupdates(): </span><span> &quot;&quot;&quot; </span><span> usage: pushconfigupdates </span><span> </span><span> Run the pushconfigupdates tool on a system. </span><span>Warning! Running PushConfigUpdates.sh deploys changes to bb-config.properties! </span><span>* This will result in an outage to the host(s) on which it is run! </span><span>* Be careful that bb-config.properties, and the xythos.properties configuration </span><span>files point to the correct database before you run this! </span><span> &quot;&quot;&quot; </span><span> sudo(&#39;/data/blackboard/tools/admin/PushConfigUpdates.sh&#39;) </span><span> </span><span>@rnt_fsm_atomic_exc </span><span>def _compress_and_delete(path, fileglob, zipage=7, rmage=3660): </span><span> &quot;&quot;&quot; </span><span> This will compress logs up to 7 days, and delete older than 62 days. </span><span> </span><span> The pattern is taken as: </span><span> </span><span> /a/b*/c/d.*.txt </span><span> </span><span> This is passed to find which will carry out the actions as sudo. </span><span> &quot;&quot;&quot; </span><span> with settings(warn_only=True): </span><span> sudo(&quot;find {path} -mtime +{zipage} -name &#39;{fileglob}&#39; -exec gzip &#39;{{}}&#39; \;&quot;.format(path=path, fileglob=fileglob, zipage=zipage)) </span><span> sudo(&quot;find {path} -mtime +{rmage} -name &#39;{fileglob}.gz&#39; -exec rm &#39;{{}}&#39; \;&quot;.format(path=path, fileglob=fileglob, rmage=rmage)) </span><span> </span><span>@task </span><span>@rnt_help </span><span>@rnt_fsm_atomic_exc </span><span>def rotate_tomcat_logs(): </span><span> &quot;&quot;&quot; </span><span> usage: rotate_tomcat_logs -&gt; atomic </span><span> </span><span> This will rotate the tomcat logs in /data/blackboard/logs/tomcat. </span><span> &quot;&quot;&quot; </span><span> log_blackboard(level=&quot;NOTICE&quot;) </span><span> with settings(warn_only=True): </span><span> for pattern in [&#39;stdout-stderr-*.log&#39;, &#39;bb-access-log.*.txt&#39;, </span><span> &#39;activemq.txt.*.txt&#39;, &#39;catalina-log.txt.*.txt&#39;, &#39;gc.*.txt&#39;, </span><span> &#39;thread_dump*.txt&#39;, &#39;*.hprof&#39; ]: </span><span> _compress_and_delete(&quot;/data/blackboard/logs/tomcat/&quot;, pattern) </span></code></pre> Render errors on websites Fri, 25 Jul 2014 00:00:00 +0000 Unknown https://fy.blackhats.net.au/blog/2014-07-25-render-errors-on-websites/ https://fy.blackhats.net.au/blog/2014-07-25-render-errors-on-websites/ <h1 id="render-errors-on-websites">Render errors on websites</h1> <p>some websites always give me weird rendering, such as % signs for buttons etc. I have finally looked into this, and realised I'm missing a font set. To fix this, just do:</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>yum install entypo-fonts </span></code></pre> NSS-OpenSSL Command How to: The complete list. Thu, 10 Jul 2014 00:00:00 +0000 Unknown https://fy.blackhats.net.au/blog/2014-07-10-nss-openssl-command-how-to-the-complete-list/ https://fy.blackhats.net.au/blog/2014-07-10-nss-openssl-command-how-to-the-complete-list/ <h1 id="nss-openssl-command-how-to-the-complete-list">NSS-OpenSSL Command How to: The complete list.</h1> <p>I am sick and tired of the lack of documentation for how to actually use OpenSSL and NSS to achieve things. Be it missing small important options like &quot;subjectAltNames&quot; in nss commands or openssls cryptic settings. Here is my complete list of everything you would ever want to do with OpenSSL and NSS.</p> <p>References:</p> <ul> <li><a href="http://www.mozilla.org/projects/security/pki/nss/tools/certutil.html">certutil mozilla</a>.</li> <li><a href="https://developer.mozilla.org/en-US/docs/NSS_reference/NSS_tools_:_certutil">nss tools</a>.</li> <li><a href="https://www.openssl.org/docs/apps/openssl.html">openssl</a>.</li> </ul> <h2 id="nss-specific">Nss specific</h2> <h2 id="db-creation-and-basic-listing">DB creation and basic listing</h2> <p>Create a new certificate database if one doesn't exist (You should see key3.db, secmod.db and cert8.db if one exists). :</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>certutil -N -d . </span></code></pre> <p>List all certificates in a database :</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>certutil -L -d . </span></code></pre> <p>List all private keys in a database :</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>certutil -K -d . [-f pwdfile.txt] </span></code></pre> <p>I have created a password file, which consists of random data on one line in a plain text file. Something like below would suffice. Alternately you can enter a password when prompted by the certutil commands. If you wish to use this for apache start up, you need to use pin.txt :</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>echo &quot;Password&quot; &gt; pwdfile.txt </span><span>echo &quot;internal:Password&quot; &gt; pin.txt </span></code></pre> <h2 id="importing-certificates-to-nss">Importing certificates to NSS</h2> <p>Import the signed certificate into the requesters database.</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>certutil -A -n &quot;Server-cert&quot; -t &quot;,,&quot; -i nss.dev.example.com.crt -d . </span></code></pre> <p>Import an openSSL generated key and certificate into an NSS database. :</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>openssl pkcs12 -export -in server.crt -inkey server.key -out server.p12 -name Test-Server-Cert </span><span>pk12util -i server.p12 -d . -k pwdfile.txt </span></code></pre> <h2 id="importing-a-ca-certificate">Importing a CA certificate</h2> <p>Import the CA public certificate into the requesters database. :</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>certutil -A -n &quot;CAcert&quot; -t &quot;C,,&quot; -i /etc/pki/CA/nss/ca.crt -d . </span></code></pre> <h2 id="exporting-certificates">Exporting certificates</h2> <p>Export a secret key and certificate from an NSS database for use with openssl. :</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>pk12util -o server-export.p12 -d . -k pwdfile.txt -n Test-Server-Cert </span><span>openssl pkcs12 -in server-export.p12 -out file.pem -nodes </span></code></pre> <p>Note that file.pem contains both the CA cert, cert and private key. You can view just the private key with: :</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>openssl pkcs12 -in server-export.p12 -out file.pem -nocerts -nodes </span></code></pre> <p>Or just the cert and CAcert with :</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>openssl pkcs12 -in server-export.p12 -out file.pem -nokeys -nodes </span></code></pre> <p>You can easily make ASCII formatted PEM from here.</p> <h2 id="both-nss-and-openssl">Both NSS and OpenSSL</h2> <h2 id="self-signed-certificates">Self signed certificates</h2> <p>Create a self signed certificate.</p> <p>For nss, note the -n, which creates a &quot;nickname&quot; (And should be unique) and is how applications reference your certificate and key. Also note the -s line, and the CN options. Finally, note the first line has the option -g, which defines the number of bits in the created certificate. :</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>certutil -S -f pwdfile.txt -d . -t &quot;C,,&quot; -x -n &quot;Server-Cert&quot; -g 2048\ </span><span>-s &quot;CN=nss.dev.example.com,O=Testing,L=example,ST=South Australia,C=AU&quot; </span><span> </span><span>openssl req -x509 -newkey rsa:2048 -keyout key.pem -out cert.pem -days </span><span> </span><span>certutil -S -f pwdfile.txt -d . -t &quot;C,,&quot; -x -n &quot;Server-Cert2&quot; \ </span><span>-s &quot;CN=nss2.dev.example.com,O=Testing,L=example,ST=South Australia,C=AU&quot; </span></code></pre> <h2 id="subjectaltnames">SubjectAltNames</h2> <p>To add subject alternative names, use a comma seperated list with the option -8 IE: :</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>certutil -S -f pwdfile.txt -d . -t &quot;C,,&quot; -x -n &quot;Server-Cert&quot; -g 2048\ </span><span>-s &quot;CN=nss.dev.example.com,O=Testing,L=example,ST=South Australia,C=AU&quot; \ </span><span>-8 &quot;nss.dev.example.com,nss-alt.dev.example.com&quot; </span></code></pre> <p>For OpenSSL this is harder:</p> <p>First, you need to create an altnames.cnf :</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>[req] </span><span>req_extensions = v3_req </span><span>nsComment = &quot;Certificate&quot; </span><span>distinguished_name = req_distinguished_name </span><span> </span><span>[ req_distinguished_name ] </span><span> </span><span>countryName = Country Name (2 letter code) </span><span>countryName_default = AU </span><span>countryName_min = 2 </span><span>countryName_max = 2 </span><span> </span><span>stateOrProvinceName = State or Province Name (full name) </span><span>stateOrProvinceName_default = South Australia </span><span> </span><span>localityName = Locality Name (eg, city) </span><span>localityName_default = example/streetAddress=Level </span><span> </span><span>0.organizationName = Organization Name (eg, company) </span><span>0.organizationName_default = example </span><span> </span><span>organizationalUnitName = Organizational Unit Name (eg, section) </span><span>organizationalUnitName_default = TS </span><span> </span><span>commonName = Common Name (eg, your name or your server\&#39;s hostname) </span><span>commonName_max = 64 </span><span> </span><span>[ v3_req ] </span><span> </span><span># Extensions to add to a certificate request </span><span> </span><span>basicConstraints = CA:FALSE </span><span>keyUsage = nonRepudiation, digitalSignature, keyEncipherment </span><span>subjectAltName = @alt_names </span><span> </span><span>[alt_names] </span><span>DNS.1 = server1.yourdomain.tld </span><span>DNS.2 = mail.yourdomain.tld </span><span>DNS.3 = www.yourdomain.tld </span><span>DNS.4 = www.sub.yourdomain.tld </span><span>DNS.5 = mx.yourdomain.tld </span><span>DNS.6 = support.yourdomain.tld </span></code></pre> <p>Now you run a similar command to before with: :</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>openssl req -x509 -newkey rsa:2048 -keyout key.pem -out cert.pem -days -config altnames.cnf </span><span>openssl req -key key.pem -out cert.csr -days -config altnames.cnf -new </span></code></pre> <h2 id="check-a-certificate-belongs-to-a-specific-key">Check a certificate belongs to a specific key</h2> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>openssl rsa -noout -modulus -in client.key | openssl sha1 </span><span>openssl req -noout -modulus -in client.csr | openssl sha1 </span><span>openssl x509 -noout -modulus -in client.crt | openssl sha1 </span></code></pre> <h2 id="view-a-certificate">View a certificate</h2> <p>View the cert :</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>certutil -L -d . -n Test-Cert </span><span> </span><span>openssl x509 -noout -text -in client.crt </span></code></pre> <p>View the cert in ASCII PEM form (This can be redirected to a file for use with openssl)</p> <p>::</p> <p>: certutil -L -d . -n Test-Cert -a certutil -L -d . -n Test-Cert -a &gt; cert.pem</p> <h2 id="creating-a-csr">Creating a CSR</h2> <p>In a second, seperate database to your CA.</p> <p>Create a new certificate request. Again, remember -8 for subjectAltName :</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>certutil -d . -R -o nss.dev.example.com.csr -f pwdfile.txt \ </span><span>-s &quot;CN=nss.dev.example.com,O=Testing,L=example,ST=South Australia,C=AU&quot; </span></code></pre> <p>Using openSSL create a server key, and make a CSR :</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>openssl genrsa -out client.key 2048 </span><span>openssl req -new -key client.key -out client.csr </span></code></pre> <h2 id="self-signed-ca">Self signed CA</h2> <p>Create a self signed CA (In a different database from the one used by httpd.) :</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>certutil -S -n CAissuer -t &quot;C,C,C&quot; -x -f pwdfile.txt -d . \ </span><span>-s &quot;CN=ca.nss.dev.example.com,O=Testing,L=example,ST=South Australia,C=AU&quot; </span></code></pre> <p>OpenSSL is the same as a self signed cert. :</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>openssl req -x509 -newkey rsa:2048 -keyout key.pem -out cert.pem -days </span></code></pre> <h2 id="signing-with-the-ca">Signing with the CA</h2> <p>Create a certificate in the same database, and sign it with the CAissuer certificate. :</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>certutil -S -n Test-Cert -t &quot;,,&quot; -c CAissuer -f pwdfile.txt -d . \ </span><span>-s &quot;CN=test.nss.dev.example.com,O=Testing,L=example,ST=South Australia,C=AU&quot; </span></code></pre> <p>If from a CSR, review the CSR you have recieved. :</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>/usr/lib[64]/nss/unsupported-tools/derdump -i /etc/httpd/alias/nss.dev.example.com.csr </span><span>openssl req -inform DER -text -in /etc/httpd/alias/nss.dev.example.com.csr ## if from nss </span><span>openssl req -inform PEM -text -in server.csr ## if from openssl </span></code></pre> <p>On the CA, sign the CSR. :</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>certutil -C -d . -f pwdfile.txt -i /etc/httpd/alias/nss.dev.example.com.csr \ </span><span>-o /etc/httpd/alias/nss.dev.example.com.crt -c CAissuer </span></code></pre> <p>For openssl CSR, note the use of -a that allows an ASCII formatted PEM input, and will create and ASCII PEM certificate output. :</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>certutil -C -d . -f pwdfile.txt -i server.csr -o server.crt -a -c CAissuer </span><span> </span><span>### Note, you may need a caserial file ... </span><span>openssl x509 -req -days 1024 -in client.csr -CA root.crt -CAkey root.key -out client.crt </span></code></pre> <h2 id="check-validity-of-a-certificate">Check validity of a certificate</h2> <p>Test the new cert for validity as an SSL server. This assumes the CA cert is in the DB. (Else you need openssl or to import it) :</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>certutil -V -d . -n Test-Cert -u V </span><span> </span><span>openssl verify -verbose -CAfile ca.crt client.crt </span></code></pre> <h2 id="export-the-ca-certificate">Export the CA certificate</h2> <p>Export the CA public certificate :</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>certutil -L -d . -n CAissuer -r &gt; ca.crt </span></code></pre> <h2 id="nss-sqlite-db">NSS sqlite db</h2> <p>Finally, these commands all use the old DBM formatted NSS databases. To use the new &quot;shareable&quot; sqlite formatting, follow the steps found from <a href="https://blogs.oracle.com/meena/entry/what_s_new_in_nss">this blog post</a>.</p> <p>How to upgrade from cert8.db to cert9.db</p> <p>You can either use environment variables or use sql: prefix in database directory parameter of certutil:</p> <p>::</p> <p>: $export NSS_DEFAULT_DB_TYPE=sql $certutil -K -d /tmp/nss -X</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>&gt; OR </span><span> </span><span>\$certutil -K -d sql:/tmp/nss -X </span></code></pre> <p>When you upgrade these are the files you get</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>key3.db -&gt; key4.db </span><span>cert8.db -&gt; cert9.db </span><span>secmod.db -&gt; pkcs11.txt </span></code></pre> <p>The contents of the pkcs11.txt files are basically identical to the contents of the old secmod.db, just not in the old Berkeley DB format. If you run the command &quot;$modutil -dbdir DBDIR -rawlist&quot; on an older secmod.db file, you should get output similar to what you see in pkcs11.txt.</p> <p>What needs to be done in programs / C code</p> <p>Either add environment variable NSS_DEFAULT_DB_TYPE &quot;sql&quot;</p> <p>NSS_Initialize call in <a href="https://developer.mozilla.org/en/NSS_Initialize">https://developer.mozilla.org/en/NSS_Initialize</a> takes this &quot;configDir&quot; parameter as shown below.</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>NSS_Initialize(configDir, &quot;&quot;, &quot;&quot;, &quot;secmod.db&quot;, NSS_INIT_READONLY); </span></code></pre> <p>For cert9.db, change this first parameter to &quot;sql:&quot; + configDir (like &quot;sql:/tmp/nss/&quot;) i.e. prefix &quot;sql:&quot; in the directory name where these NSS Databases exist. This code will work with cert8.db as well if cert9.db is not present.</p> <p><a href="https://wiki.mozilla.org/NSS_Shared_DB">https://wiki.mozilla.org/NSS_Shared_DB</a></p> <h2 id="display-a-human-readable-certificate-from-an-ssl-socket">Display a human readable certificate from an SSL socket</h2> <p>Note: port 636 is LDAPS, but all SSL sockets are supported. For TLS only a limited set of protocols are supported. Add -starttls to the command. See man 1 s_client.</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>openssl s_client -connect ldap.example.com:636 </span><span> </span><span>[ant@ant-its-example-edu-au ~]$ echo -n | openssl s_client -connect ldap.example.com:636 | sed -ne &#39;/-BEGIN CERTIFICATE-/,/-END CERTIFICATE-/p&#39; | openssl x509 -noout -text </span><span> </span><span>depth=3 C = SE, O = AddTrust AB, OU = AddTrust External TTP Network, CN = AddTrust External CA Root </span><span>verify return:1 </span><span>depth=2 C = US, ST = UT, L = Salt Lake City, O = The USERTRUST Network, OU = http://www.usertrust.com, CN = UTN-USERFirst-Hardware </span><span>verify return:1 </span><span>depth=1 C = AU, O = AusCERT, OU = Certificate Services, CN = AusCERT Server CA </span><span>verify return:1 </span><span>depth=0 C = AU, postalCode = 5000, ST = South Australia, L = example, street = Level, street = Place, O =Example, OU = Technology Services, CN = ldap.example.com </span><span>verify return:1 </span><span>DONE </span><span>Certificate: </span><span> Data: </span><span> Version: 3 (0x2) </span><span> Serial Number: </span><span> Signature Algorithm: sha1WithRSAEncryption </span><span> Issuer: C=AU, O=AusCERT, OU=Certificate Services, CN=AusCERT Server CA </span><span> Validity </span><span> Not Before: XX </span><span> Not After : XX </span><span> Subject: C=AU/postalCode=5000, ST=South Australia, L=example/street=Level /street=Place, O=Example, OU=Technology Services, CN=ldap.example.com </span><span> Subject Public Key Info: </span><span>&lt;snip&gt; </span><span> X509v3 Subject Alternative Name: </span><span> DNS:ldap.example.com </span><span>&lt;snip&gt; </span></code></pre> <p>You can use this to display a CA chain if you can't get it from other locations.</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>openssl s_client -connect ldap.example.com:636 -showcerts </span></code></pre> <h2 id="mod-nss">mod_nss</h2> <p>To configure mod_nss, you should have a configuration similar to below - Most of this is the standard nss.conf that comes with mod_nss, but note the changes to NSSNickname, and the modified NSSPassPhraseDialog and NSSRandomSeed values. There is documentation on the NSSCipherSuite that can be found by running &quot;rpm -qd mod_nss&quot;. Finally, make sure that apache has read access to the database files and the pin.txt file. If you leave NSSPassPhraseDialog as &quot;builtin&quot;, you cannot start httpd from systemctl. You must run apachectl so that you can enter the NSS database password on apache startup.</p> <p>NOTE: mod_nss <em>DOES NOT</em> support SNI.</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>LoadModule nss_module modules/libmodnss.so </span><span>Listen 8443 </span><span>NameVirtualHost *:8443 </span><span>AddType application/x-x509-ca-cert .crt </span><span>AddType application/x-pkcs7-crl .crl </span><span>NSSPassPhraseDialog file:/etc/httpd/alias/pin.txt </span><span>NSSPassPhraseHelper /usr/sbin/nss_pcache </span><span>NSSSessionCacheSize 10000 </span><span>NSSSessionCacheTimeout 100 </span><span>NSSSession3CacheTimeout 86400 </span><span>NSSEnforceValidCerts off </span><span>NSSRandomSeed startup file:/dev/urandom 512 </span><span>NSSRenegotiation off </span><span>NSSRequireSafeNegotiation off </span><span>&lt;VirtualHost *:8443&gt; </span><span>ServerName nss.dev.example.com:8443 </span><span>ServerAlias nss.dev.example.com </span><span>ErrorLog /etc/httpd/logs/nss1_error_log </span><span>TransferLog /etc/httpd/logs/nss1_access_log </span><span>LogLevel warn </span><span>NSSEngine on </span><span>NSSProtocol TLSv1 </span><span>NSSNickname Server-cert </span><span>NSSCertificateDatabase /etc/httpd/alias </span><span>&lt;Files ~ &quot;\.(cgi|shtml|phtml|php3?)$&quot;&gt; </span><span> NSSOptions +StdEnvVars </span><span>&lt;/Files&gt; </span><span>&lt;Directory &quot;/var/www/cgi-bin&quot;&gt; </span><span> NSSOptions +StdEnvVars </span><span>&lt;/Directory&gt; </span><span>&lt;/VirtualHost&gt; </span></code></pre> Linux remote desktop from GDM Wed, 19 Jun 2013 00:00:00 +0000 Unknown https://fy.blackhats.net.au/blog/2013-06-19-linux-remote-desktop-from-gdm/ https://fy.blackhats.net.au/blog/2013-06-19-linux-remote-desktop-from-gdm/ <h1 id="linux-remote-desktop-from-gdm">Linux remote desktop from GDM</h1> <p>For quite some time I have wanted to be able to create thin linux workstations that automatically connect to a remote display manager of some kind for the relevant desktop services. This has always been somewhat of a mystery to me, but I found the final answer to be quite simple.</p> <p>First, you need a system like a windows Remote Desktop server, or xrdp server configured. Make sure that you can connect and login to it.</p> <p>Now install your thin client. I used CentOS with a minimal desktop install to give me an X server.</p> <p>Install the &quot;rdesktop&quot; package on your thin client.</p> <p>Now you need to add the Remote Desktop session type.</p> <p>Create the file &quot;/usr/bin/rdesktop-session&quot; (Or /opt or /srv. Up to you - but make sure it's executable)</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>#!/bin/bash </span><span>/usr/bin/rdesktop -d domain.example.com -b -a 32 -x lan -f termserv.example.com </span></code></pre> <p>Now you need to create a session type that GDM will recognise. Put this into &quot;/usr/share/xsessions/rdesktop.desktop&quot;. These options could be improved etc.</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>[Desktop Entry] </span><span>Name=RDesktop </span><span>Comment=This session logs you into RDesktop </span><span>Exec=/usr/bin/rdesktop-session </span><span>TryExec=/usr/bin/rdesktop-session </span><span>Terminal=True </span><span>Type=Application </span><span> </span><span>[Window Manager] </span><span>SessionManaged=true </span></code></pre> <p>Create a user who will automatically connect to the TS.</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>useradd remote_login </span></code></pre> <p>Configure GDM to automatically login after a time delay. The reason for the time delay, is so that after the rdesktop session is over, at the GDM display, a staff member can shutdown the thin client.</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>[daemon] </span><span>TimedLoginEnable=True </span><span>TimedLogin=remote_login </span><span>TimedLoginDelay=15 </span></code></pre> <p>Finally, set the remote login user's session to RDesktop &quot;/home/remote_login/.dmrc&quot;</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>[Desktop] </span><span>Session=rdesktop </span></code></pre> <p>And that's it!</p> <p>If you are using windows terminal services, you will notice that the login times out after about a minute, GDM will reset, wait 15 seconds and connect again, causing a loop of this action. To prevent this, you should extend the windows server login timeout. On the terminal server:</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Control\Terminal Server\WinStations\[[Connection endpoint]]\LogonTimeout (DWord, seconds for timeout) </span></code></pre> <p>[[Connection endpoint]] is the name in RD Session Host configurations : I had rebuilt mine as default and was wondering why this no longer worked. This way you can apply the logon timeout to different session connections.</p> <p>Update: Actually, it needs to be RDP-Tcp regardless of the connection endpoint. Bit silly.</p> Akonadi mariadb on ZFS Fri, 24 May 2013 00:00:00 +0000 Unknown https://fy.blackhats.net.au/blog/2013-05-24-akonadi-mariadb-on-zfs/ https://fy.blackhats.net.au/blog/2013-05-24-akonadi-mariadb-on-zfs/ <h1 id="akonadi-mariadb-on-zfs">Akonadi mariadb on ZFS</h1> <p>I have recently bit the bullet and decided to do some upgrades to my laptop. The main focus was getting ZFS as my home drive.</p> <p>In doing so Akonadi, the PIM service for kmail broke.</p> <p>After some investigation, it is because zfs does not support AIO with maria db.</p> <p>To fix this add to ~/.local/share/akonadi/myself.conf :</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>innodb_use_native_aio=0 </span></code></pre> MBP b43 wireless Thu, 02 May 2013 00:00:00 +0000 Unknown https://fy.blackhats.net.au/blog/2013-05-02-mbp-b43-wireless/ https://fy.blackhats.net.au/blog/2013-05-02-mbp-b43-wireless/ <h1 id="mbp-b43-wireless">MBP b43 wireless</h1> <p>I have found recently after about 3.7 that b43 wireless with most access points is quite flakey. Thankfully, a fellow student, Kram found this great blog post about getting it to work.</p> <p><a href="http://www.rdoxenham.com/?p=317">blog here</a>.</p> <p>For the moment, you have to rebuild the module by hand on update, but it's a make, make install, dracut away.</p> <p>The only thing missed is that at the end:</p> <p>Put the blacklist options into their own wl.conf rather than the main blacklist for finding them.</p> <p>You need to rebuild your dracut image. The following should work:</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>cd /boot/ </span><span>mv initramfs-[current kernel here] initramfs-[kernel].back </span><span>dracut </span></code></pre> Changing SSSD cert Thu, 25 Apr 2013 00:00:00 +0000 Unknown https://fy.blackhats.net.au/blog/2013-04-25-changing-sssd-cert/ https://fy.blackhats.net.au/blog/2013-04-25-changing-sssd-cert/ <h1 id="changing-sssd-cert">Changing SSSD cert</h1> <p>After re-provisioning my Samba 4 domain, I found SSSD giving m a strange error: :</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>ldap_install_tls failed: [Connect error] </span><span> [TLS error -8054:You are attempting to import a cert with the same issuer/serial as an existing cert, but that is not the same cert.] </span></code></pre> <p>It seems SSSD caches the ca cert of your ldap service (even if you change the SSSD domain name). I couldn't find where to flush this, but changing some of the tls options will fix it.</p> <p>In SSSD.conf:</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>ldap_id_use_start_tls = True </span><span>ldap_tls_cacertdir = /usr/local/samba/private/tls </span><span>ldap_tls_reqcert = demand </span></code></pre> <p>Now to make the cacertdir work you need to run</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>cacertdir_rehash /usr/local/samba/private/tls </span></code></pre> <p>Your SSSD should now be working again.</p> Virtual hosted django Mon, 18 Feb 2013 00:00:00 +0000 Unknown https://fy.blackhats.net.au/blog/2013-02-18-virtual-hosted-django/ https://fy.blackhats.net.au/blog/2013-02-18-virtual-hosted-django/ <h1 id="virtual-hosted-django">Virtual hosted django</h1> <p>Recently I have been trying to host multiple django applications on a single apache instance.</p> <p>Sometimes, you would find that the page from a different vhost would load incorrectly. This is due to the way that WSGI handles work thread pools.</p> <p>To fix it.</p> <p>In your /etc/httpd/conf.d/wsgi.conf Make sure to comment out the WSGIPythonPath line. :</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>#WSGIPythonPath </span><span>WSGISocketPrefix run/wsgi </span><span>#You can add many process groups. </span><span>WSGIDaemonProcess group_wsgi python-path=&quot;/var/www/django/group&quot; </span></code></pre> <p>Now in your VHost add the line (If your script alias is &quot;/&quot;) :</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>&lt;location &quot;/&quot;&gt; </span><span>WSGIProcessGroup group_wsgi </span><span>&lt;/location&gt; </span></code></pre> Steam Linux Beta on Fedora 18 (x86 64 or x86) Fri, 07 Dec 2012 00:00:00 +0000 Unknown https://fy.blackhats.net.au/blog/2012-12-07-steam-linux-beta-on-fedora-18-x86-64-or-x86/ https://fy.blackhats.net.au/blog/2012-12-07-steam-linux-beta-on-fedora-18-x86-64-or-x86/ <h1 id="steam-linux-beta-on-fedora-18-x86-64-or-x86">Steam Linux Beta on Fedora 18 (x86 64 or x86)</h1> <p>These instructions are old! Use this instead:</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>wget http://spot.fedorapeople.org/steam/steam.repo -O /etc/yum.repos.d/steam.repo </span><span>yum install steam </span></code></pre> <p>OLD METHOD below</p> <p>Get the .deb.</p> <p>Unpack it with :</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>ar x steam.deb </span><span>tar -xvzf data.tar.gz -C / </span></code></pre> <p>Now install</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>yum install glibc.i686 \ </span><span>libX11.i686 \ </span><span>libstdc++.i686 \ </span><span>mesa-libGL.i686 \ </span><span>mesa-dri-drivers.i686 \ </span><span>libtxc_dxtn.i686 \ </span><span>libXrandr.i686 \ </span><span>pango.i686 \ </span><span>gtk2.i686 \ </span><span>alsa-lib.i686 \ </span><span>nss.i686 \ </span><span>libpng12.i686 \ </span><span>openal-soft.i686 \ </span><span>pulseaudio-libs.i686 </span></code></pre> <p>Now you should be able to run the steam client from /usr/bin/steam or from the Applications - Games menu</p> <p>If you have issues, try :</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>cd ~/.local/share/Steam </span><span>LD_DEBUG=&quot;libs&quot; ./steam.sh </span></code></pre> <p>To see what is going on. Sometimes you will see something like :</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>9228: trying file=tls/i686/sse2/libGL.so.1 </span><span>9228: trying file=tls/i686/libGL.so.1 </span><span>9228: trying file=tls/sse2/libGL.so.1 </span><span>9228: trying file=tls/libGL.so.1 </span><span>9228: trying file=i686/sse2/libGL.so.1 </span><span>9228: trying file=i686/libGL.so.1 </span><span>9228: trying file=sse2/libGL.so.1 </span><span>9228: trying file=libGL.so.1 </span><span>9228: search cache=/etc/ld.so.cache </span><span>9228: search path=/lib/i686:/lib/sse2:/lib:/usr/lib/i686:/usr/lib/sse2:/usr/lib (system search path) </span><span>9228: trying file=/lib/i686/libGL.so.1 </span><span>9228: trying file=/lib/sse2/libGL.so.1 </span><span>9228: trying file=/lib/libGL.so.1 </span><span>9228: trying file=/usr/lib/i686/libGL.so.1 </span><span>9228: trying file=/usr/lib/sse2/libGL.so.1 </span><span>9228: trying file=/usr/lib/libGL.so.1 </span></code></pre> <p>And the steam client will then hang, or say &quot;Error loading steamui.so&quot;. It is because you are missing libGL.so.1 in this case.</p> <p>running ldd against the files in &quot;.local/share/Steam/ubuntu12_32/&quot; should reveal most of the deps you need.</p> NSS commands and how to Tue, 01 May 2012 00:00:00 +0000 Unknown https://fy.blackhats.net.au/blog/2012-05-01-nss-commands-and-how-to/ https://fy.blackhats.net.au/blog/2012-05-01-nss-commands-and-how-to/ <h1 id="nss-commands-and-how-to">NSS commands and how to</h1> <p>I have collated some knowledge on how to use NSS and it's tools for some general purpose usage, including mod_nss.</p> <p>Much of this is just assembling the contents of the <a href="http://www.mozilla.org/projects/security/pki/nss/tools/certutil.html">certutil documentation</a>.</p> <p>In this I have NOT documented the process of deleting certificates, changing trust settings of existing certificates or changing key3.db passwords.</p> <p>Create a new certificate database if one doesn't exist (You should see key3.db, secmod.db and cert8.db if one exists).</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>certutil -N -d . </span></code></pre> <p>List all certificates in a database</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>certutil -L -d . </span></code></pre> <p>List all private keys in a database</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>certutil -K -d . [-f pwdfile.txt] </span></code></pre> <p>I have created a password file, which consists of random data on one line in a plain text file. Something like below would suffice. Alternately you can enter a password when prompted by the certutil commands. If you wish to use this for apache start up, you need to use pin.txt</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>echo &quot;soeihcoraiocrthhrcrcae aoriao htuathhhohodrrcrcgg89y99itantmnomtn&quot; &gt; pwdfile.txt </span><span>echo &quot;internal:soeihcoraiocrthhrcrcae aoriao htuathhhohodrrcrcgg89y99itantmnomtn&quot; &gt; pin.txt </span></code></pre> <p>Create a self signed certificate in your database. Note the -n, which creates a &quot;nickname&quot; (And should be unique) and is how applications reference your certificate and key. Also note the -s line, and the CN options. Finally, note the first line has the option -g, which defines the number of bits in the created certificate.</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>certutil -S -f pwdfile.txt -d . -t &quot;C,,&quot; -x -n &quot;Server-Cert&quot; -g 2048\ </span><span>-s &quot;CN=nss.dev.example.com,O=Testing,L=Adelaide,ST=South Australia,C=AU&quot; </span><span>certutil -S -f pwdfile.txt -d . -t &quot;C,,&quot; -x -n &quot;Server-Cert2&quot; \ </span><span>-s &quot;CN=nss2.dev.example.com,O=Testing,L=Adelaide,ST=South Australia,C=AU&quot; </span></code></pre> <p>To add subject alternative names, use a comma seperated list with the option -8 IE</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>certutil -S -f pwdfile.txt -d . -t &quot;C,,&quot; -x -n &quot;Server-Cert&quot; -g 2048\ </span><span>-s &quot;CN=nss.dev.example.com,O=Testing,L=Adelaide,ST=South Australia,C=AU&quot; \ </span><span>-8 &quot;nss.dev.example.com,nss-alt.dev.example.com&quot; </span></code></pre> <p>Create a self signed CA (In a different database from the one used by httpd.)</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>certutil -S -n CAissuer -t &quot;C,C,C&quot; -x -f pwdfile.txt -d . \ </span><span>-s &quot;CN=ca.nss.dev.example.com,O=Testing,L=Adelaide,ST=South Australia,C=AU&quot; </span></code></pre> <p>Create a certificate in the same database, and sign it with the CAissuer certificate.</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>certutil -S -n Test-Cert -t &quot;,,&quot; -c CAissuer -f pwdfile.txt -d . \ </span><span>-s &quot;CN=test.nss.dev.example.com,O=Testing,L=Adelaide,ST=South Australia,C=AU&quot; </span></code></pre> <p>Test the new cert for validity as an SSL server.</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>certutil -V -d . -n Test-Cert -u V </span></code></pre> <p>View the new cert</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>certutil -L -d . -n Test-Cert </span></code></pre> <p>View the cert in ASCII form (This can be redirected to a file for use with openssl)</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>certutil -L -d . -n Test-Cert -a </span><span>certutil -L -d . -n Test-Cert -a &gt; cert.pem </span></code></pre> <p>In a second, seperate database to your CA.</p> <p>Create a new certificate request. Again, remember -8 for subjectAltName</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>certutil -d . -R -o nss.dev.example.com.csr -f pwdfile.txt \ </span><span>-s &quot;CN=nss.dev.example.com,O=Testing,L=Adelaide,ST=South Australia,C=AU&quot; </span></code></pre> <p>On the CA, review the CSR you have recieved.</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>/usr/lib[64]/nss/unsupported-tools/derdump -i /etc/httpd/alias/nss.dev.example.com.csr </span><span>openssl req -inform DER -text -in /etc/httpd/alias/nss.dev.example.com.csr </span></code></pre> <p>On the CA, sign the CSR.</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>certutil -C -d . -f pwdfile.txt -i /etc/httpd/alias/nss.dev.example.com.csr \ </span><span>-o /etc/httpd/alias/nss.dev.example.com.crt -c CAissuer </span></code></pre> <p>Export the CA public certificate</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>certutil -L -d . -n CAissuer -r &gt; ca.crt </span></code></pre> <p>Import the CA public certificate into the requestors database.</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>certutil -A -n &quot;CAcert&quot; -t &quot;C,,&quot; -i /etc/pki/CA/nss/ca.crt -d . </span></code></pre> <p>Import the signed certificate into the requestors database.</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>certutil -A -n &quot;Server-cert&quot; -t &quot;,,&quot; -i nss.dev.example.com.crt -d . </span></code></pre> <p>Using openSSL create a server key, and make a CSR</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>openssl genrsa -out server.key 2048 </span><span>openssl req -new -key server.key -out server.csr </span></code></pre> <p>On the CA, review the CSR.</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>openssl req -inform PEM -text -in server.csr </span></code></pre> <p>On the CA, sign the request. Note the use of -a that allows an ASCII formatted PEM input, and will create and ASCII PEM certificate output.</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>certutil -C -d . -f pwdfile.txt -i server.csr -o server.crt -a -c CAissuer </span></code></pre> <p>Import an openSSL generated key and certificate into an NSS database.</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>openssl pkcs12 -export -in server.crt -inkey server.key -out server.p12 -name Test-Server-Cert </span><span>pk12util -i server.p12 -d . -k pwdfile.txt </span></code></pre> <p>Export a secret key and certificate from an NSS database for use with openssl.</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>pk12util -o server-export.p12 -d . -k pwdfile.txt -n Test-Server-Cert </span><span>openssl pkcs12 -in server-export.p12 -out file.pem -nodes </span></code></pre> <p>Note that file.pem contains both the CA cert, cert and private key. You can view just the private key with:</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>openssl pkcs12 -in server-export.p12 -out file.pem -nocerts -nodes </span></code></pre> <p>Or just the cert and CAcert with</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>openssl pkcs12 -in server-export.p12 -out file.pem -nokeys -nodes </span></code></pre> <p>You can easily make ASCII formatted PEM from here.</p> <p>Finally, these commands all use the old DBM formatted NSS databases. To use the new &quot;shareable&quot; sqlite formatting, follow the steps found from <a href="https://blogs.oracle.com/meena/entry/what_s_new_in_nss1">this blog post</a>.</p> <p>To configure mod_nss, you should have a configuration similar to below - Most of this is the standard nss.conf that comes with mod_nss, but note the changes to NSSNickname, and the modified NSSPassPhraseDialog and NSSRandomSeed values. There is documentation on the NSSCipherSuite that can be found by running &quot;rpm -qd mod_nss&quot;. Finally, make sure that apache has read access to the database files and the pin.txt file. If you leave NSSPassPhraseDialog as &quot;builtin&quot;, you cannot start httpd from systemctl. You must run apachectl so that you can enter the NSS database password on apache startup.</p> <p>NOTE: mod_nss <em>might</em> support SNI. In my testing and examples, this works to create multiple sites via SNI, however, other developers claim this is not a supported feature. I have had issues with it in other instances also. For now, I would avoid it.</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>LoadModule nss_module modules/libmodnss.so </span><span>Listen 8443 </span><span>NameVirtualHost *:8443 </span><span>AddType application/x-x509-ca-cert .crt </span><span>AddType application/x-pkcs7-crl .crl </span><span>NSSPassPhraseDialog file:/etc/httpd/alias/pin.txt </span><span>NSSPassPhraseHelper /usr/sbin/nss_pcache </span><span>NSSSessionCacheSize 10000 </span><span>NSSSessionCacheTimeout 100 </span><span>NSSSession3CacheTimeout 86400 </span><span>NSSEnforceValidCerts off </span><span>NSSRandomSeed startup file:/dev/urandom 512 </span><span>NSSRenegotiation off </span><span>NSSRequireSafeNegotiation off </span><span>&lt;VirtualHost *:8443&gt; </span><span>ServerName nss.dev.example.com:8443 </span><span>ServerAlias nss.dev.example.com </span><span>ErrorLog /etc/httpd/logs/nss1_error_log </span><span>TransferLog /etc/httpd/logs/nss1_access_log </span><span>LogLevel warn </span><span>NSSEngine on </span><span>NSSCipherSuite +rsa_rc4_128_md5,+rsa_rc4_128_sha,+rsa_3des_sha,+fips_3des_sha,+rsa_aes_128_sha,+rsa_aes_256_sha,\ </span><span>-rsa_des_sha,-rsa_rc4_40_md5,-rsa_rc2_40_md5,-rsa_null_md5,-rsa_null_sha,-fips_des_sha,-fortezza,-fortezza_rc4_128_sha,\ </span><span>-fortezza_null,-rsa_des_56_sha,-rsa_rc4_56_sha </span><span>NSSProtocol SSLv3,TLSv1 </span><span>NSSNickname Server-cert </span><span>NSSCertificateDatabase /etc/httpd/alias </span><span>&lt;Files ~ &quot;\.(cgi|shtml|phtml|php3?)$&quot;&gt; </span><span> NSSOptions +StdEnvVars </span><span>&lt;/Files&gt; </span><span>&lt;Directory &quot;/var/www/cgi-bin&quot;&gt; </span><span> NSSOptions +StdEnvVars </span><span>&lt;/Directory&gt; </span><span>&lt;/VirtualHost&gt; </span><span>&lt;VirtualHost *:8443&gt; </span><span>ServerName nss2.dev.example.com:8443 </span><span>ServerAlias nss2.dev.example.com </span><span>ErrorLog /etc/httpd/logs/nss2_error_log </span><span>TransferLog /etc/httpd/logs/nss2_access_log </span><span>LogLevel warn </span><span>NSSEngine on </span><span>NSSCipherSuite +rsa_rc4_128_md5,+rsa_rc4_128_sha,+rsa_3des_sha,+fips_3des_sha,+rsa_aes_128_sha,+rsa_aes_256_sha,\ </span><span>-rsa_des_sha,-rsa_rc4_40_md5,-rsa_rc2_40_md5,-rsa_null_md5,-rsa_null_sha,-fips_des_sha,-fortezza,-fortezza_rc4_128_sha,\ </span><span>-fortezza_null,-rsa_des_56_sha,-rsa_rc4_56_sha </span><span>NSSProtocol SSLv3,TLSv1 </span><span>NSSNickname Server-Cert2 </span><span>NSSCertificateDatabase /etc/httpd/alias </span><span>&lt;Files ~ &quot;\.(cgi|shtml|phtml|php3?)$&quot;&gt; </span><span> NSSOptions +StdEnvVars </span><span>&lt;/Files&gt; </span><span>&lt;Directory &quot;/var/www/cgi-bin&quot;&gt; </span><span> NSSOptions +StdEnvVars </span><span>&lt;/Directory&gt; </span><span>&lt;/VirtualHost&gt; </span></code></pre> Slow mac sleep Thu, 26 Apr 2012 00:00:00 +0000 Unknown https://fy.blackhats.net.au/blog/2012-04-26-slow-mac-sleep/ https://fy.blackhats.net.au/blog/2012-04-26-slow-mac-sleep/ <h1 id="slow-mac-sleep">Slow mac sleep</h1> <p>Recently, I have noticed that my shiny macbook pro 8,2, with 16GB of ram and it's super fast intel SSD, was taking quite a long time to sleep - near 20 seconds to more than a minute in some cases. This caused me frustration to no avail.</p> <p>However, recently, in an attempt to reclaim disk space from the SSD, in the form of a wasted 16GB chunk in /private/var/vm/sleepimage . This lead me to read the documentation on pmutil.</p> <p>hibernate mode is set to 3 by default - this means that when you close the lid on your MBP, it dumps the contents of ram to sleepimage, and then suspends to ram. This means in the case that you lose power while suspended, you can still restore your laptop state safely. I don't feel I need this, so I ran the following. :</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>sudo pmset -a hibernatemode 0 </span><span>sudo rm /private/var/vm/sleepimage </span></code></pre> <p>Now I have saved 16GB of my SSD (And read write cycles) and my MBP sleeps in 2 seconds flat.</p> Samba 4 Internal DNS use Mon, 16 Apr 2012 00:00:00 +0000 Unknown https://fy.blackhats.net.au/blog/2012-04-16-samba-4-internal-dns-use/ https://fy.blackhats.net.au/blog/2012-04-16-samba-4-internal-dns-use/ <h1 id="samba-4-internal-dns-use">Samba 4 Internal DNS use</h1> <p>It took me a while to find this in an email from a mailing list.</p> <p>To use the internal DNS from samba4 rather than attempting to use BIND9 append the line &quot;--dns-backend=SAMBA_INTERNAL&quot; to your provision step.</p> Mod Selinux with Django Sun, 15 Apr 2012 00:00:00 +0000 Unknown https://fy.blackhats.net.au/blog/2012-04-15-mod-selinux-with-django/ https://fy.blackhats.net.au/blog/2012-04-15-mod-selinux-with-django/ <h1 id="mod-selinux-with-django">Mod Selinux with Django</h1> <p>Django with mod_selinux</p> <p>The mod_selinux module allows you to confine a spawned apache process into a specific selinux context. For example, you can do this via virtual hosts, or by LocationMatch directives.</p> <p>Part of my curiosity wanted to see how this works. So I made up a small django application that would tell you the selinux context of an URL.</p> <p>Install mod_selinux first</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>yum install mod_selinux mod_wsgi </span></code></pre> <p>Now we create a VirtualHost that we can use for the test application</p> <p>:</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>NameVirtualHost *:80 </span><span> </span><span>&lt;VirtualHost *:80&gt; </span><span> ServerAdmin william@firstyear.id.au </span><span> DocumentRoot /var/empty </span><span> ServerName 172.16.209.150 </span><span> </span><span> &lt;LocationMatch /selinux/test/c2&gt; </span><span> selinuxDomainVal *:s0:c2 </span><span> &lt;/LocationMatch&gt; </span><span> &lt;LocationMatch /selinux/test/c3&gt; </span><span> selinuxDomainVal *:s0:c3 </span><span> &lt;/LocationMatch&gt; </span><span> </span><span> #Alias /robots.txt /usr/local/wsgi/static/robots.txt </span><span> #Alias /favicon.ico /usr/local/wsgi/static/favicon.ico </span><span> </span><span> AliasMatch ^/([^/]*\.css) /var/www/django_base/static/styles/$1 </span><span> </span><span> Alias /media/ /var/www/django_base/media/ </span><span> Alias /static/ /var/www/django_base/static/ </span><span> </span><span> &lt;Directory /var/www/django_base/static&gt; </span><span> Order deny,allow </span><span> Allow from all </span><span> &lt;/Directory&gt; </span><span> </span><span> &lt;Directory /var/www/django_base/media&gt; </span><span> Order deny,allow </span><span> Allow from all </span><span> &lt;/Directory&gt; </span><span> </span><span> WSGIScriptAlias / /var/www/django_base/django_base/wsgi.py </span><span> </span><span> &lt;Directory /var/www/django_base/scripts&gt; </span><span> Order allow,deny </span><span> Allow from all </span><span> &lt;/Directory&gt; </span><span>&lt;/VirtualHost&gt; </span></code></pre> <p>We also need to alter /etc/httpd/conf.d/mod_selinux.conf to have MCS labels.</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>selinuxServerDomain *:s0:c0.c100 </span></code></pre> <p>And finally, download the (now sadly lost) tar ball, and unpack it to /var/www</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>cd /var/www </span><span>tar -xvzf django_selinux_test.tar.gz </span></code></pre> <p>Now, navigating to the right URL will show you the different SELinux contexts</p> <p><a href="http://localhost/selinux/test/test">http://localhost/selinux/test/test</a> :</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>Hello. Your processes context is [0, &#39;system_u:system_r:httpd_t:s0:c0.c100&#39;] </span></code></pre> <p><a href="http://localhost/selinux/test/c2">http://localhost/selinux/test/c2</a> :</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>Hello. Your processes context is [0, &#39;system_u:system_r:httpd_t:s0:c2&#39;] </span></code></pre> <p><a href="http://localhost/selinux/test/c3">http://localhost/selinux/test/c3</a> :</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>Hello. Your processes context is [0, &#39;system_u:system_r:httpd_t:s0:c3&#39;] </span></code></pre> <p>The best part about this is that this context is passed via the local unix socket to sepgsql - meaning that specific locations in your Django application can have different SELinux MCS labels, allowing mandatory access controls to tables and columns. Once I work out row-level permissions in sepgsql, these will also be available to django processes via this means.</p> <p>Example of why you want this.</p> <p>You have a shopping cart application. In your users profile page, you allow access to that URL to view / write to the credit card details of a user. In the main application, this column is in a different MCS - So exploitation of the django application, be it SQL injection, or remote shell execution - the credit cards remain in a separate domain, and thus inaccessible.</p> <p>Additionally, these MCS labels are applied to files uploaded into /media for example, so you can use this to help restrict access to documents etc.</p> SEPGSQL - How to Fedora 16 - 17 Sun, 15 Apr 2012 00:00:00 +0000 Unknown https://fy.blackhats.net.au/blog/2012-04-15-sepgsql-how-to-fedora-16-17/ https://fy.blackhats.net.au/blog/2012-04-15-sepgsql-how-to-fedora-16-17/ <h1 id="sepgsql-how-to-fedora-16-17">SEPGSQL - How to Fedora 16 - 17</h1> <p>First, we install what we will be using.</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>yum install postgresql postgresql-server postgresql-contrib </span></code></pre> <p>First, we want to setup sepgsql. sepgsql.so is part of the contrib package. These modules are installed on a per database basis, so we need to initdb first</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>postgresql-setup initdb </span></code></pre> <p>Edit vim /var/lib/pgsql/data/postgresql.conf +126</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>shared_preload_libraries = &#39;sepgsql&#39; # (change requires restart) </span></code></pre> <p>Now, we need to re-label all the default postgres tables.</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>su postgres </span><span>export PGDATA=/var/lib/pgsql/data </span><span>for DBNAME in template0 template1 postgres; do postgres --single -F -c exit_on_error=true $DBNAME /dev/null; done </span><span>exit </span></code></pre> <p>Now we can start postgresql.</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>systemctl start postgresql.service </span></code></pre> <p>Moment of truth - time to find out if we have selinux contexts in postgresql.</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span># su postgres </span><span># psql -U postgres postgres -c &#39;select sepgsql_getcon();&#39; </span><span>could not change directory to &quot;/root&quot; </span><span> sepgsql_getcon </span><span>------------------------------------------------------- </span><span> unconfined_u:unconfined_r:unconfined_t:s0-s0:c0.c1023 </span><span>(1 row) </span></code></pre> <p>We can create a new database. Lets call it setest. We also add an apache user for the django threads to connect to later. Finally, we want to setup password authentication, and change ownership of the new setest db to apache.</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>createdb setest </span><span>createuser </span><span>Enter name of role to add: apache </span><span>Shall the new role be a superuser? (y/n) n </span><span>Shall the new role be allowed to create databases? (y/n) n </span><span>Shall the new role be allowed to create more new roles? (y/n) n </span><span>psql -U postgres template1 -c &quot;alter user apache with password &#39;password&#39;&quot; </span><span>psql -U postgres template1 -c &quot;alter user postgres with password &#39;password&#39;&quot; </span><span>psql -U postgres template1 -c &quot;alter database setest owner to apache&quot; </span></code></pre> <p>Now we change our auth in postgres to be md5 in the file $PGDATA/pg_hdb.conf</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span># &quot;local&quot; is for Unix domain socket connections only </span><span>local all all md5 </span><span># IPv4 local connections: </span><span>host all all 127.0.0.1/32 md5 </span><span># IPv6 local connections: </span><span>host all all ::1/128 md5 </span><span> </span><span>systemctl restart postgresql.service </span></code></pre> <p>Now you should be able to login in with a password as both users.</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span># psql -U postgres -W </span><span>Password for user postgres: </span><span>psql (9.1.3) </span><span>Type &quot;help&quot; for help. </span><span> </span><span>postgres=# </span><span># psql -U apache -W setest </span><span>Password for user apache: </span><span>psql (9.1.3) </span><span>Type &quot;help&quot; for help. </span><span> </span><span>setest=# </span></code></pre> <p>Lets also take this chance, to take a look at the per column and per table selinux permissions.</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>psql -U postgres -W setest -c &quot;SELECT objtype, objname, label FROM pg_seclabels WHERE provider = &#39;selinux&#39; AND objtype in (&#39;table&#39;, &#39;column&#39;)&quot; </span></code></pre> <p>To update these</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>SECURITY LABEL FOR selinux ON TABLE mytable IS &#39;system_u:object_r:sepgsql_table_t:s0&#39;; </span></code></pre> <p>See <a href="http://www.postgresql.org/docs/9.1/static/sql-security-label.html">also</a>.</p> <p>This is very useful, especially if combined with my next blog post.</p> DHCP6 server Mon, 22 Aug 2011 00:00:00 +0000 Unknown https://fy.blackhats.net.au/blog/2011-08-22-dhcp6-server/ https://fy.blackhats.net.au/blog/2011-08-22-dhcp6-server/ <h1 id="dhcp6-server">DHCP6 server</h1> <p>I have been battling with setting up one of these for a long time. It so happens most areas of the internet, forget to mention one vital piece of the DHCP6 puzzle - DHCP6 is not standalone. It is an addition to RADVD. Thus you need to run both for it to work correctly.</p> <p>Why would you want DHCP6 instead of RADVD? Well, RADVD may be good for your simple home use with a few computers, and MDNS name resoultion. But when you look at a business, a LAN party, or those who want DDNS updates, it is essential.</p> <p>First, we need to setup RADVD properly. The order of these directives is <em>very</em> important.</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>interface eth0 </span><span>{ </span><span> AdvManagedFlag on; </span><span> AdvOtherConfigFlag on; </span><span> AdvSendAdvert on; </span><span> MinRtrAdvInterval 5; </span><span> MaxRtrAdvInterval 60; </span><span> prefix 2001:db8:1234:4321/64 </span><span> { </span><span> AdvOnLink on; </span><span> AdvAutonomous on; </span><span> AdvRouterAddr on; </span><span> }; </span><span>}; </span></code></pre> <p>Next, we need to configure DHCP6. I am using the ISC-DHCP4 server. DHCP6 needs its own instance. Fedora provides a seperate script for this (dhcpd6.service) that you can use. On other OSes' you may not have this and will need to start DHCPD manually with the -6 flag. Here is the config you need.</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>server-name &quot;server.example.com&quot; ; </span><span>server-identifier server.example.com ; </span><span> </span><span>authoritative; </span><span>option dhcp6.name-servers 2001:db8:1234:4321::1 ; </span><span>ddns-update-style interim ; </span><span>ddns-domainname &quot;example.com&quot;; </span><span> </span><span>subnet6 2001:db8:1234:4321::/64 { </span><span> range6 2001:db8:1234:4321::10 2001:db8:1234:4321::110 ; </span><span>} </span></code></pre> <p>Now, since DHCP6 uses UDP / TCP (Its layer 3, and runs across link local), you must consider your firewall. On both client and server you need to accept icmp6, port 546 and 547 from the following addresses</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>Server: </span><span>Source - fe80::/16 </span><span>Destination - ff02::1:2 </span><span> </span><span>Client </span><span>Source - ff02::1:2 </span><span>Source - fe80::/16 </span><span>Destination - fe80::/16 </span></code></pre> <p>A set of example iptables rules on the server side would be</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>-A INPUT -p ipv6-icmp -j ACCEPT </span><span>-A INPUT -s fe80::/16 -d ff02::1:2 -i eth0 -p udp -m udp --dport 546 -j ACCEPT </span><span>-A INPUT -s fe80::/16 -d ff02::1:2 -i eth0 -p tcp -m tcp --dport 546 -j ACCEPT </span><span>-A INPUT -s fe80::/16 -d ff02::1:2 -i eth0 -p udp -m udp --dport 547 -j ACCEPT </span><span>-A INPUT -s fe80::/16 -d ff02::1:2 -i eth0 -p tcp -m tcp --dport 547 -j ACCEPT </span></code></pre> <p>And similar enough for the client.</p> <p>Now start radvd, dhcp6 and your firewalls. Then on your client run. Enjoy your DHCP6! :</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>dhclient -d -v -6 interface </span></code></pre> <p>From here, it is very similar to DHCP4 to add things like DDNS updates to your DHCP6 server.</p> Mod auth cas Sun, 10 Jul 2011 00:00:00 +0000 Unknown https://fy.blackhats.net.au/blog/2011-07-10-mod-auth-cas/ https://fy.blackhats.net.au/blog/2011-07-10-mod-auth-cas/ <h1 id="mod-auth-cas">Mod auth cas</h1> <p>Recently at UofA, It was mentioned in passing &quot;Wouldn't it be nice to have <a href="http://www.jasig.org/cas">CAS</a> auth on the webserver instead of ldap basic auth?&quot;.</p> <p>&quot;Yes, It would be &quot;, I said. But it got me thinking about the issues involved. While nice to use CAS, CAS only provides authentication, not authorization. We rely on ldap attributes for determining access to content.</p> <p>After a few minutes of reading, I found the solution.</p> <h2 id="installation">Installation</h2> <p>I tested this on CentOS 5 (As we use RHEL at work), so adjust this for your needs.</p> <p>If EPEL is not enabled you can enable it with this</p> <p><a href="http://fedoraproject.org/wiki/EPEL">EPEL</a>.</p> <p>If you wish to only install the one package, you can set the repository to disabled, and install with the following command</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>yum install --enablerepo=epel mod_auth_cas </span></code></pre> <p>Also install the ldap module. It is part of the base repo in RHEL.</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>yum install mod_authz_ldap </span></code></pre> <h2 id="configuration">Configuration</h2> <p>Stop your apache server</p> <p>We need the modules to load in a certain order, so we need to rename our configs.</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>cd /etc/httpd/conf.d/ </span><span>mv auth_cas.conf 00_auth_cas.conf </span><span>mv authz_ldap.conf 10_authz_ldap.conf </span><span>mv ssl.conf 20_ssl.conf </span></code></pre> <p>In /etc/httpd/conf.d/00_auth_cas.conf</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span># </span><span># mod_auth_cas is an Apache 2.0/2.2 compliant module that supports the </span><span># CASv1 and CASv2 protocols </span><span># </span><span> </span><span>LoadModule auth_cas_module modules/mod_auth_cas.so </span><span>&lt;IfModule mod_auth_cas.c&gt; </span><span> CASVersion 2 </span><span> CASDebug On </span><span> </span><span> # Validate the authenticity of the login.goshen.edu SSL certificate by </span><span> # checking its chain of authority from the root CA. </span><span> CASCertificatePath /etc/pki/tls/certs/ </span><span> CASValidateServer Off </span><span> CASValidateDepth 9 </span><span> </span><span> CASCookiePath /var/lib/cas/ </span><span> </span><span> CASLoginURL https://auth.example.com/cas/login </span><span> CASValidateURL https://auth.example.com/cas/serviceValidate </span><span> CASTimeout 7200 </span><span> CASIdleTimeout 7200 </span><span> &lt;/IfModule&gt; </span></code></pre> <p>DO NOT RESTART APACHE YET.</p> <p>You need to create the cas tickets directory, else the module will barf.</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>cd /var/lib </span><span>sudo mkdir cas </span><span>sudo chown apache:apache cas </span><span>sudo chmod 750 cas </span><span>sudo semanage fcontext -a -s system_u -t httpd_var_lib_t /var/lib/cas </span><span>sudo restorecon -r -v ./ </span></code></pre> <p>This applies the needed SELinux policy to allow httpd to write to that directory. If you have set SELinux to permissive or disabled, these steps are worth taking incase you enable SELinux again in the future.</p> <p>&lt;strong&gt;Configuration with LDAP authorization&lt;/strong&gt;</p> <p>Now we can add our ldap attributes we need. Check that 10_authz_ldap.conf matches the following</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span># </span><span># mod_authz_ldap can be used to implement access control and </span><span># authenticate users against an LDAP database. </span><span># </span><span> </span><span>LoadModule authz_ldap_module modules/mod_authz_ldap.so </span><span> </span><span>&lt;IfModule mod_authz_ldap.c&gt; </span><span> </span><span>## Some commented code </span><span> </span><span>&lt;/IfModule&gt; </span></code></pre> <p>Now, in your SSL Directory directive add</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>&lt;Directory &quot;/var/www/ms1&quot;&gt; </span><span> Order allow,deny </span><span> Allow from all </span><span> AuthType CAS </span><span> AuthName &quot;TEST CAS AUTH&quot; </span><span> AuthLDAPURL ldaps://ldap.example.com:636/ou=People,dc=example,dc=com?uid?one? </span><span> require ldap-filter &amp;(uid=username) </span><span> &lt;/Directory&gt; </span></code></pre> <p>You can start apache again after reading the filter section</p> <h2 id="filter">Filter</h2> <p>This ldap filter can be anything you desire. It can be a list of UID's, sets of attributes, etc.</p> <p>examples:</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>#Will check for this attribute </span><span>&amp;(department=marketing) </span><span>#Checks that one has both this class and this department </span><span>&amp;(class=compsci1001)(department=marketing) </span><span>#Your name is either foo or bar </span><span>|(uid=foo)(uid=bar) </span><span>#These can be nested as well. This would allow anyone with attr and other attr OR the uid= foo into the site. </span><span>|(&amp;((attr=true)(other attr=true)) (uid=foo)) </span></code></pre> <p>You can read more about filters <a href="http://www.zytrax.com/books/ldap/apa/search.html">here</a>.</p> <p>Alternately, one can change the configuration to be like so</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>AuthLDAPURL ldaps://ldap.example.com:636/ou=People,dc=example,dc=com?uid?one?(&amp;(attr=foo)(attr=bar)) </span><span>Require valid-user </span></code></pre> <p>Note the filters are the same, but require the whole filter to be enclosed in a set of ().</p> SELinux for postfix + dovecot Tue, 05 Jul 2011 00:00:00 +0000 Unknown https://fy.blackhats.net.au/blog/2011-07-05-selinux-for-postfix-dovecot/ https://fy.blackhats.net.au/blog/2011-07-05-selinux-for-postfix-dovecot/ <h1 id="selinux-for-postfix-dovecot">SELinux for postfix + dovecot</h1> <blockquote> <p>I am currently in the middle of creating an email solution for the doctors surgery that I work for. I have previously tried exchange, but found it to slow, and unreliable for my needs. Instead, I have decided to go with postfix + dovecot for the doctors needs.</p> </blockquote> <p>In my experimenting, I have been using a fedora VM, with SElinux enabled. However, SELinux has decided to hate on everything I do for this, and thus in my inability to accept defeat, I have created an SELinux module that should allow postfix and dovecot to work as per following <a href="http://www.1a-centosserver.com/centos_linux_mail_server/centos_mail_server.php">this email setup guide</a>.</p> <p>the module is</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>module postfixmysql 1.0; </span><span> </span><span>require { </span><span> type mysqld_var_run_t; </span><span> type postfix_map_t; </span><span> type usr_t; </span><span> type mysqld_t; </span><span> type mysqld_db_t; </span><span> type postfix_virtual_t; </span><span> type postfix_smtpd_t; </span><span> type postfix_cleanup_t; </span><span> class sock_file write; </span><span> class unix_stream_socket connectto; </span><span> class file getattr; </span><span> class dir search; </span><span>} </span><span> </span><span>#============= postfix_cleanup_t ============== </span><span>allow postfix_cleanup_t mysqld_db_t:dir search; </span><span>allow postfix_cleanup_t mysqld_t:unix_stream_socket connectto; </span><span>allow postfix_cleanup_t mysqld_var_run_t:sock_file write; </span><span>allow postfix_cleanup_t usr_t:file getattr; </span><span> </span><span>#============= postfix_map_t ============== </span><span>allow postfix_map_t mysqld_db_t:dir search; </span><span>allow postfix_map_t mysqld_t:unix_stream_socket connectto; </span><span>allow postfix_map_t mysqld_var_run_t:sock_file write; </span><span> </span><span>#============= postfix_smtpd_t ============== </span><span>allow postfix_smtpd_t mysqld_db_t:dir search; </span><span>allow postfix_smtpd_t mysqld_t:unix_stream_socket connectto; </span><span>allow postfix_smtpd_t mysqld_var_run_t:sock_file write; </span><span> </span><span>#============= postfix_virtual_t ============== </span><span>allow postfix_virtual_t mysqld_db_t:dir search; </span><span>allow postfix_virtual_t mysqld_t:unix_stream_socket connectto; </span><span>allow postfix_virtual_t mysqld_var_run_t:sock_file write; </span></code></pre> <p>This can be built and installed with a command like such (as root)</p> <pre style="background-color:#2b303b;color:#c0c5ce;"><code><span>checkmodule -M -m -o postfixmysql.mod postfixmysql.te; semodule_package -m postfixmysql.mod -o postfixmysql.pp; semodule -i postfixmysql.pp </span></code></pre>