Austin Janey

@ajaney

Sysadmin by day and IT consultant by night. I drink lots of coffee and solve lots of problems... hopefully.

6,138 words

austinjaney.com Guestbook
You'll only receive email when Austin Janey publishes a new post

Replacing Windows File Servers with CentOS 7

After a fair amount of trial and error I finally have a process thats working well for me.  This is in no way a comprehensive guide on using SSSD with Samba to authenticate active directory users/groups to file shares but its a great start and is working well in my lab.  Many thanks to all those who contributed to articles in the helpful resources list at the bottom.

Part 1: Install and configure SSSD
Packages needed for SSSD to work correctly
yum install realmd sssd adcli oddjob oddjob-mkhomedir samba-common-tools net-tools ntpdate ntp

Network Configuration
make sure you have a network connection, if you installed the above packages then you should be good.
Edit your network configuration:
vi /etc/sysconfig/network
Edit your hosts file:
vi /etc/hosts
Restart networking
/etc/init.d/network restart

setup system time
systemctl enable ntpd.service
ntpdate yourdomaincontroller.yourdomain.tld
systemctl start ntpd.service
NOTE: some have noted that in order for things to work right you might need to add your DC as a server entry to /etc/ntp.conf, I have not yet needed to do this.

Join the domain
sudo realm join -v -U domainuser addomainname.com
You can use either the ID command against a user or use realm list to discover if you have joined the domain.

Considerations
  • Once you are domain joined anyone on the domain can SSH into the joined server. 
  • You may want to lock down your sudoers policy

SSH Config
In order to limit what users are allowed to login to the newly joined server you will want to edit your ssh config
 /etc/ssh/sshd_config

Add the lines:
NOTE: Adding an ad group to control ssh permissions is a good idea, if you were to add the group ssh-users in AD you would add the line:
  • don't assume group nesting will work, SSSD only looks at the immediate users of a group.
  • NOTE: Doing this will explicitly allow only members of domain group you listed to log in.

sudoers file
this is not the best or least privilege way to do this but it is the way that will allow you to control everything in AD, create a group in AD that you want to give sudoers rights to and add the following line to your sudoers file on your newly joined server.  
Traditionally, the visudo command opens the /etc/sudoers file with the vi text editor.
%groupname@ADDOMAIN.COM ALL=(ALL:ALL) ALL
  • caps may be required for the domain name.

Part 2: Install and configure SAMBA
Install Samba
Yum install samba

make sure samba can talk threw the firewall
firewall-cmd --permanent --add-service=samba
firewall-cmd --reload

smb.conf working example
The following samba config file was pulled from a working server.
# See smb.conf.example for a more detailed config file or
# read the smb.conf manpage.
# Run 'testparm' to verify the config is correct after
# you modified it.
[global]
        workgroup = YOURDOMAINNAMEWITHNOTLD
        server string = Samba Server Version %v
        encrypt passwords = yes
        security = ads
        realm = REPLACEWITHYOURDOMAINNAME
        passdb backend = tdbsam
        printing = cups
        printcap name = cups
        load printers = yes
        cups options = raw
        kerberos method = secrets and keytab
        load printers = no
        cups options = raw
        printcap name = /dev/null
        log file = /var/log/samba/log.%m
        max log size = 50
#Test fix for idmap bug
        idmap config * : backend = tdb
        idmap config * : range = 300000-400000
[home directory]
        path = /home/%u
        comment = Home Directories
        guest ok = no
        browseable = yes
        read only = no
        inherit acls = yes
        inherit permissions = yes
        valid users = @“SOMEGROUP@YOURDOMAIN.TLD"
        admin users = @"SOMEGROUP@YOURDOMAIN.TLD"
[printers]
        comment = All Printers
        path = /var/tmp
        printable = Yes
        create mask = 0600
        browseable = No
[print$]
        comment = Printer Drivers
        path = /var/lib/samba/drivers
        write list = @printadmin root
        force group = @printadmin
        create mask = 0664
        directory mask = 0775

An example of what look to me like some sane defaults from http://www.hexblot.com/blog/centos-7-active-directory-and-samba includes:
[global]
workgroup = MYDOMAINLOCAL
server string = Samba Server Version %v

# Add the IPs / subnets allowed acces to the server in general.
# The following allows local and 10.0.*.* access
hosts allow = 127. 10.0.

# log files split per-machine:
log file = /var/log/samba/log.%m
# enable the following line to debug:
# log level =3
# maximum size of 50KB per log file, then rotate:
max log size = 50

# Here comes the juicy part!
security = ads
encrypt passwords = yes
passdb backend = tdbsam
realm = MYDOMAIN.LOCAL

# Not interested in printers
load printers = no
cups options = raw

# This stops an annoying message from appearing in logs
printcap name = /dev/null

Now that samba is setup to share /home youll need to edit permissions on /home so users can access their home folders.  In the case of active directory domain home folders using “domain users@yourdomain.tld” should provide a good option.
chown root:"adgroupyoumade@yourdomain.tld" /home
chmod 0770 /home

Note about SELinux:
If you haven’t disabled it (which you probably shouldn’t) Upon finishing up and setting permissions you might find that you can’t access your shares, it might be SELinux. You either need to
  • (Please don't) disable it completely (by setting SELINUX=disabled in /etc/sysconfig/selinux ) or
  • enter the following command for each share you make: 
chcon -t samba_share_t /var/myshare
To share out home directories you will need to run
setsebool -P samba_enable_home_dirs on

Enable and Start up Samba
systemctl enable smb.service
systemctl start smb.service

Congrats you should now be able to authenticate to your samba file shares using active directory authentication! 

Helpful resources
Notes on SE Linus and best practice: https://wiki.centos.org/HowTos/SetUpSamba
Notes on integrating with AD (huge thanks to Hexblot) http://www.hexblot.com/blog/centos-7-active-directory-and-samba

The Stages of Security Awareness

More and more business, institutions, and Individuals are willing to reason that the cost of a data breach is less than or equal to the cost of treating customer data with the same care they treat their own social security number, email password, or bank information. Or at least it seems that way up until the point at which they get caught or become aware that they didn't invest in securing business data. Like many things, the problem is becoming aware of the issue. So what are the key points of failure in becoming aware?  What keeps business, institutions, and Individuals from securing important data?  These are the attitudes I commonly see, they align themselves nicely with the first 5 stages of grief.

1. Shock: But we don't have anything of value to steal?  We couldn't be a target.

There are a couple things in this one that make it interesting.  First, it assumes that securing data is about data theft, data theft however is rather benign, what its used for is what causes the damage.  All businesses run on data, criminals have a business of their own and they exclusively want your data to be able to use it.  By knowing things about you (sometimes even in real time) an attacker can know what you are doing, corporate espionage is a real thing.  Depend on their access method they may also be able to appear as you. The other part of this is question implies that data might be the only thing an attacker wants to steal, a computer connected to the internet might be just as valuable depending on what their purposes are.  Brian Krebs has an excellent article on this.

2. Denial: We've never had a data breach before.

My favorite quip here is the classic "Well I've never died before, so maybe I never will!" This is perhaps the worst posture and often held by people who fundamentally don't understand technology or that humans get better at any given task over time, including crime. On the flip side, how do you know you have never had a data breach? Often it's the case that people who give this as some kind of reason for not investing in a security strategy are the same people who don't have firewall logs, adequate antivirus, any way to manage workstations or firewalls that haven't seen a firmware patch in over 5 years.

3. Anger: But who would want to attack us?

This one is interesting, the answer might be nobody.  Even if the answer is nobody the problem is that most business that has experienced some type of cyber security incident also had that answer.  The nature of cybersecurity today happens to be that attacks are not specifically targeted, in other words, your company is not important to an attacker and they don't care about you.  You are the complete opposite of special.  To the attacker, your email address just appeared in a leaked list or database of addresses, or maybe it was in a mailbox or address list of someone who already got hacked.  Its likely if your an older company or organization that you have some email addresses sitting out there, check have i been pwned it's better to know sooner rather than later.  Nobody is probably targeting you or your organization but somebody is always targeting everyone, targeting everyone is much easier than targeting someone.

4. Bargaining: Having security is inconvenient and slows business down I only want to secure things that don't have an impact.

This is in some cases true, treating customer data correctly does mean in some cases that extra care must be taken to ensure it is managed properly.  Being a sysadmin is a lot like being a private butler in that regard, your IT staff are stewards of the precious data your company uses to make money, so are your accountants, your salespeople and your janitors (yep, they probably have access to every unlocked workstation at night when they are cleaning).  If you only secure things that don't have an impact than the only things you will secure are things you're not impacted by.

5. Depression: There's no way to secure everything and criminals getting into our systems in inevitable.

Cool, go ahead and put that on your website. This is the most dangerous attitude and companies that have it when found out typically don't end up doing business anymore. The price of IT security is eternal vigilance. 

Jurassic Park

Jurassic Park is an excellent sci-fi movie it checks all of the right boxes:
Genetic engineering? Check.
John Williams? Check.
Samuel L Jackson saying "Hold onto your butts" with a cigarette in is mouth? Check.
Possibly evil mega-corporation bringing back dinosaurs and thus monetizing life itself? Check.
A subplot about a bunch of computers going offline to be fixed by some kid who says “it's a Unix system I know this!”? Check.
Humans running for their lives from dinosaurs? Check.
Lawyer caught on the toilet by a T-Rex? Double Check.

Something easy to overlook however is that it's equally filled with warnings about systems automation, a foreboding tale about corporate espionage/sabotage, and how poor business continuity planning brings about disasters that can at least, in this case, can literally eat you alive.

Welcome to Jurassic Park, a post-mortem about workplace culture problems and how they affect IT, workers.
Many people probably think the disasters on Isla Nublar were caused by rogue programmer Dennis Nedry, But truth be told Jurassic Park had much bigger problems. Even if Dennis had never decided to steal dinosaur embryos in a modified Barbasol bottle Jurassic Park still likely would have been doomed. Why? Culture, that's why.  In the case of InGen, Jurassic Park, and Dennis Nedry: “He had become very annoyed with InGen; Hammond continually asked for things which hadn't been included in his original contract, and InGen demanded that they are done. When Nedry refused, lawsuits were threatened and letters were written to Nedry's other clients insinuating that he was unreliable. Nedry had no other choice but to return and carry out the extra work, but for no extra money.” -jurassicpark.wikia.com

Dennis clearly had his reasons to hate his employer, who had no problem manipulating him just like they did the DNA of dinosaurs.  The problems created in IT departments because of culture mismanagement are huge, in the case of Jurassic Park those problems were:
  • Reduced size of IT staff created a dependence on undocumented automated processes.
    • Note: When Sysadmins are busy and alone they don’t document anything, in this case, Dennis just didn’t care but worth keeping in mind, if documentation is your priority and your sysadmins can’t seem to do that it's probably not because they don’t care.  Sysadmins are typically nerds and nerds love to brag about how cool the thing they just built is, sysadmins do this through documentation, its a personal priority for anyone who takes that role seriously, so if its not getting done odds are its because management hasn’t provided IT with enough “Free Time” to do so.  If your a manager check yourself before you wreck yourself.
  • Inability to step away pinned the stability of Jurassic Park to Dennis instead of an IT team.
  • Lack of properly configured logging and alerting systems, Dennis should never have been able to run code against the security system without it first being checked by teammates.  Source control is important.

Problems like these are not isolated however and are often felt across infrastructure engineering as well, engineering issues such as:
  • Lack of redundant systems (Ironically those systems that should have been there to prevent a T-Rex from eating the company lawyer, that T-Rex might be a subtle allegory for ransomware or other system calamities)
  • Lack of adequate logging/alerting systems for the systems Dennis was responsible for.
  • Lack of antivirus systems or properly configured workstations, even though what Dennis wrote was not technically malware if a sysadmin can write code for your systems without it being authorized and signed through a code signing process it mine as well be.  Having some form of application/code whitelisting such as app locker is critical in the modern enterprise.

So how do you prevent a Jurassic Park incident? Culture, and IT culture specifically The mark of a good company culture is that it creates the ability to step away for a time with little to no business impact felt, furthermore it should encourage it. The ability to step away is likewise the trademark of a good Sysadmin.  If you think the main goal of a sysadmin is to keep all the computers working, servers humming, and network up you would be sorely mistaken even more so if you are a sysadmin and you think you have job security because that's how you define your job. Dennis’s exact problem was that his employer had made it his job and pushed moving targets on top of it. The main goal of a sysadmin (or any employee) should be to achieve the results of the company they work for.  A good sysadmin should be a driver of positive change that accelerates businesses to meet and excel past their goals, this is where job security comes from.  If I step away from my job (which I have, I’m writing this in a coffee shop in Maui Hawaii) I have full faith that my coworkers will be able to use the systems I've set up to keep everything running.
  • Our passwords are in a password manager
  • Our primary systems are redundant with backups
  • Our network is designed to reasonably handle internet outages and gateway failures
  • We have antivirus and reporting squared away
  • There are a variety of workstations I have left prepared to be deployed in case of hardware failure or in case my process for provisioning new workstations is not understood quickly enough to meet an immediate need.
  • In the event that my plane crashes in the Pacific and I’m eaten by sharks the documentation I’ve left behind should be informative enough for another sysadmin to do my job and meet business needs quickly.

Thought this was cool: http://jurassicsystems.com/about

Digital security for normal people

I was in Starbucks the other day and overheard a local computer tech helping someone reinstall windows on their laptop, the tech left and I started a conversation with the laptop owner.  His laptop had been infected with ransomware and he, unfortunately, didn't have a backup. We had a short conversation about backups where the painfully obvious was stated and not much more.  Having backups may not sound like a security strategy but that's because many people think that security is about protecting yourself from bad guys and internet scams. Security is not about protecting yourself from "hackers" cyber criminals, malware or online scams, it encompasses a much wider practice. Security is the art of protecting time. In the case of the man I met at Starbucks what he had lost was documents that he spends time writing, pictures he had spent time taking, bookmarks he had spent time finding, business data he had spent time working on, and a computer he now had to spend time getting fixed using money he had spent time earning. 

Correctly thinking about security depends on what you are trying to protect, for most people at a minimum that means their own time, for people like my self who have chosen IT as a carrier that means protecting the time of others as well.  To best do that its important to have a working definition of what security means. I define security as:

"Security is the art of protecting assets, knowledge or time in such a way that the “Cost” of destroying, disrupting, or disappearing them is insurmountably high. The “Cost” of attack is equal to the amount of either negligence or effort that you or an attacker must take to destroy, disrupt, or disappear the protected asset. Realistically if someone is able to pay the “Cost” in either time or money to conduct the attack they can compromise your security."

The following is the collection of advice I wish I could have also given him but just did not have the time to, this is also advice I give to family members, coworkers, and people like you who stumble across my website.  This is how you increase the cost of an attack.


A. Securing Online Accounts 

  1. Use a password manager and avoid reusing passwords across sites like the plague, side note: it is the plague.  LastPass and 1password are a great starting point. There are likely many other good online options.  In my opinion, the most important thing about a password manager is that it be zero knowledge, meaning that the company running the service your using has no way to decrypt the data you entrust them to store. If you don’t like the idea of storing your passwords online take a look at offline options such as KeePass, password safe, or perfect paper passwords.
  2. Enable 2nd-factor authentication on all your accounts, especially your chosen password manager.
  3. Setup haveibeenpwned.com for the email account/s you use.
  4. Recognize the human error factor, humans make mistakes. When you use the web make sure you're using an adblocker to avoid malicious advertisements that might lead you to a phishing site.  Ublock Origin is great for this.  Using 3rd party DNS is also a great help, Quad9 or OpenDNS Greatly increases your security at no cost and is fairly easy to set up on your router or computer.

B. Securing The Personal Computer

  1. Don’t use an admin account for every day computing this applies to macOS, Linux, and Windows no exceptions.  Follow the Principle of least privilege.
  2. Data security is just as important as account security in most cases, having backups is the best way to secure your data from accidental deletion, corruption, and ransomware. Veeam endpoint free is free and does a great job backing up your entire system.
  3. Run an up to date version of your operating system and preferred web browser and ensure you have security updates installed.
  4. If your computer does get infected just nuke and pave.  If your system has been compromised it truly is the only way to be sure your safe again.  Make sure you have a good backup, erase the internal disk, and reinstall your operating system.

A note on Antivirus Software: I did not mention antivirus here for the reason that consumer-grade antivirus systems seem to change like the wind lately.  In general, if you're looking for an antivirus system I would recommend looking at reviews from IT people as they will spend a lot more time than you can imagine looking at antivirus solutions for their respective companies.  Nearing the end of 2017 I had begun to see a rise in malware that exploits antivirus systems to compromise the systems they were designed to protect, in general, your best antivirus option is having an up to date computer with the most recent security patches installed and following best practices, B.1 is your best bet.

C. Securing The Data

  1. 3-2-1 Backups,  If your data is not following 3-2-1 backups your data does not exist.  Make sure you can restore your backups.
  2. If your storing sensitive data in the cloud use some form of “pre-internet encryption” for windows, mac and Linux veracrypt is probably the golden standard but there are other encryption tools, even having an encrypted zip file is better than nothing.  Note: password protected and encrypted are different things. Know the difference and use the right one.
  3. Back up everything. If its unimportant data back it up, if its important data back it up again.  The number one reason important data cant be restored is that someone didn’t think it was important and thus did not back it up.  If you backup everything all the time this is an easy pitfall to avoid.

D. Securing The Network

  1. If your router can be found at routerpwn.com consider getting a different router or looking for firmware updates the fix the issue listed. If your router does not have firmware updates or a fix for a known issue then its time to get a different router.
  2. Take a look at what GRC’s ShieldsUP! has to say, if your router has open ports make sure you have NAT enabled on your router. The best option to avoid potential conflict is to simply not be there “True Stealth” is the result you want from the ShieldsUP! test.
  3. If you have internet of things devices on your network use the 3 Dumb Routers method to separate out your network.
  4. If you have WiFi make sure you're using a good password, only use WPA2 or greater authentication and disable WPS if possible.
  5. Use a 3rd party DNS server on your router Quad9 or OpenDNS are good options. To find out what DNS server is the quickest around you run the DNS Name Speed Benchmark from GRC.com
  6. If you don’t require devices in your wireless network to talk to each other (this is rare) or have particular devices that don’t need to talk to other devices for any reason consider putting those devices on your guest network.  Doing so will isolate those devices from the rest of your network making them less risky.

E. Securing the Human

This is the hardest part, even if you have done everything else correctly we are only human and are going to mess something up. Securing the human part of the system comes down to checking yourself as you use your technology. There are a lot of moving parts to this but in general, the following are true and if followed will make you less of a risk to yourself.

  1. Always Go to the Source, if you receive a phone call from your bank and they want to verify your social security number over the phone just hang up, Google your bank's phone number (or look on the back of your debit card) and call your bank.  If it truly was them then your good to go, if it wasn't congratulations you have just evaded an attack. The same applies to handling email phishing messages. A common email I've seen is a message warning that your inbox is about to run out of space. If you click the link it then prompts you to login to your cloud email. The right thing to do is ask your email admin if you are running out of space or go to the source and find out if you are approaching a space limit. By going to the source almost all phishing attacks can be thwarted.  
  2. TNO, Trust No One. Criminals don’t target computer systems they target people. Be cautious about giving out information. Well designed systems and services shouldn't require you to have any trust in the people running them for your data to be safe.
  3. If it's too good to be true… (you know the rest of this one, your mother told you, my mother told me, the attacker's mother told him we all know this.) SPOILER ALERT: it isThere is no Indian prince willing his inheritance to you and there is no free iPad you won.  There is always a phishing campaign in the works run by smart people who are looking to make you the sucker.  Think about the cost of a phishing message, how much it cost you to send an email?  Right… if it only costs the bad guy a couple minutes of their time to try and cheat people out of their money then guess what they are going to try and do. Furthermore, attackers have reduced the cost of an attack by using automation.  The result of this is that its no longer a couple minutes per person phished its a couple minutes per millions, and its target is not you… its target is everyone.


Resource List

Final Thoughts: We live in a world now where hackers are driving the cost of attacking systems down by having systems and automation do the attacks for them. Microsoft said it best I think “Eternal vigilance is the price of security”.

Enabling DKIM in Office 365

Enabling DKIM in office 365 is way harder than it should be if you're not letting Microsoft manage your DNS records for you.  Office 365 is not my preferred groupware system but it seems to be a necessary evil in the business world.  In order to enable DKIM for office 365 its required that you add two CNAME records.  In the example below I'm using the domain name azulpine.com and the office 365 tenet azulpine.com.  If you have a .com TLD then it follows you should be able to drop in your domain in place of where I have put azulpine. 

Cname Record 1
Host:selector1._domainkey.azulpine.com.
Value:selector1-azulpine-com._domainkey.azulpine.onmicrosoft.com

Cname Record 2
Host:selector2._domainkey.azulpine.com.
Value:selector2-azulpine-com._domainkey.azulpine.onmicrosoft.com

Why is DKIM important? 

DKIM is a way for you to sign outbound messages automatically with a special seal of approval. This is the digital equivalent of a fancy wax seal on a letter.  By doing this you can prove to an email server that the message came from an authorized source. In your DMARC record, you can specify what you would like mail servers to do with unsigned messages just like you can with messages that don’t meet SPF.  DKIM exists to provide extra assurance of a message's origin over what SPF can deliver and can reduce the likelihood of a message being marked as spam in some cases but is generally only taken into account if the receiving mail server is using DMARC to vet messages.  Aside from being used in conjunction with DMARC, DKIM can also be used to prove a message was a spoof.  If you have a loose SPF policy and are not using DMARC (which you really should) it may be a good idea to sign email messages using DKIM so that if someone impersonates your email address and sends a malicious email to one of your contacts, you have a way to prove (via your “wax seal”) to that person that you did not send a malicious email.  Damage might have been done but at least if you have DKIM you can point to the fact that you were innocent of sending a malicious message.