*NIX Journey: TL;DR Version

Newnix's *NIX Journey (TL;DR-ed)

I thought it'd be fun to write up my journey from first failed installs to where I am now, with no desire to use a more complex installer than OpenBSD or DragonFly BSD, prefering to instead run through the installation manually. Hopefully it can be useful to someone just starting out on their *NIX journey, since Windows and OSX/MacOS can be used without ever having to know anything at all about the underlying system, while with Linux, BSD, Illumos, and other "alternative" Operating Systems this sort of knowlege can be vital for getting help from online communities.

I'm hoping to provide some documentation that can help newcomers work through this sort of information without the time crunch of their drives failing or their system failing to boot. This shouldn't be mistaken for a technical guide or a CLI primer, rather this is just meant to be a record of how to get familiar with your system setup and how to determine how you feel about the different aspects, possibly even allow you to find the means to correct it or customize it to suit you better.


The Blunder Years

So I initially got into alternative Operating Systems back in high school, around 2010. Unlike most people of my generation, I actually didn't have any internet access until around freshman year, and didn't have my own computer until a couple years earlier. So things like the Commodore and other personal computers of that timeframe never entered into my life. I loved the idea of FLOSS software and Operating Systems, people coming together to create an Operating System for free, in their free time? Sign me up!

Well, looking into how to get started, I came across Debian and Ubuntu, not really knowing anything other than what I found on Wikipedia, I decided to go with Debian because of the "non-free" software in Ubuntu. Little did I know that meant the code was proprietary or distributed under a restrictive license, not necessarily that the software was paid for. Also, the tagline of "The Universal Operating System" really appealed to me. So I wound up getting a Debian DVD and failed multiple times to install it. The same thing happened with Ubuntu. Turns out my problem was that I didn't know the password that had been captured during install, but that was enough for me to give up for the time being, with no idea where to get help, I figured that either something was wrong with my systems, or I was just doing something wrong.


Intro to Linux

When I finally got a distro installed (by this time, Zorin OS), in 2013-2014, I got into an interesting situation where I could feel comfortable with the terminal. At this time I was primarily only using it to compile and run C++ code that I was working on for school. But it did spark an interest in what packages are, how to manage them, and how to use the command line to get things done.

As time went on, I eventually went on to try more up-to-date distros like Sabayon, where I got more acquainted with the concept of package management. Before long I thought I was an expert and decided I should tackle Arch, after all, Arch is tough, and is only for "hardcore" Linux users, right? Turns out it's nowhere near as tough and educational as people frequently proclaim. Eventually I moved on to Gentoo, and loved the control I had over the system, and that's where I was happy for about 2yrs.


Filesystem Experimentation

As I learned more about these systems I love, I found that there's several filesystems available, other than EXT4. While for most people, EXT4 is fine (much like people using Windows are frequently unaware of the NTFS shortcomings, and similar for OSX/MacOS users never being exposed to HFS/HFS+ issues), but I'm the kind of person that constantly has to be tinkering on something. There's various options:

* JFS
* XFS
* EXT2
* EXT3
* EXT4
* F2FS
* BTRFS
* vFAT
* ReiserFS
* Reiser4
* HAMMER
* HAMMER2
* ZFS
* BCACHEFS

Not all of them are good ideas for your system, but that'll be covered in the longer post. Tinkering with these filesystems pulled me away from Linux, because Linux doesn't actually support all of them. I highly reccommend looking into these different filesystems, as they have several interesting options and features that can be useful in various situations.


Beyond Linux

Linux is a hell of a gateway drug, after getting into a reasonable flow and being comfortable with Linux as my daily driver, I found other fun Operating Systems doing some really cool things. To list them in no particular order:

* Plan 9
* FreeBSD
* TrueOS
* DragonFly BSD
* OpenBSD
* NetBSD
* Haiku
* 9front
* Open Indiana
* SmartOS 
* ReactOS
* RedoxOS
* HardenedBSD

There's a whole world of cool things happening, Linux is just the most popular. Digging into these really gives you a lot of exposure into how these things work and how the different systems are doing things differently between them.


Current Day

So today, I'm very happy with DragonFly BSD, I'm not too sure what exactly attracts me to this project over others, but it's one of the systems that works reliably on my hardware and lets me get my work done. HAMMER and HAMMER2 are just additional benefits that make it fantastic. I also make heavy use of OpenBSD and HardenedBSD in my work, primarily because of the security hardening benefits. Linux is fantastic, BSD is terrific, and once you spend some time digging into the systems you come across, you can find all sorts of interesting projects going on, even get started on your own.

Site's NOT Dead

No, I've not abandoned this site.

I've not abandoned this site, I've just been stuck in a somewhat strange state of being too busy to actually write up any new posts due to things like work, school, and trying to get in shape.
I've actually even had a bit of a change of heart too, changing from using Linux exclusively to using various forms of BSD exclusively, it's just impossible to resist the call of those awesome filesystems, like HAMMER and ZFS.
I'm now working full time as a salaried employee in the network management department of my company. Time is still likely going to be a factor in how frequently I can crank out new posts, but if you like what I have, there should be an RSS feed available to notify you when I actually get around to publishing these things. I have no intention of making any money from this site, so don't worry about ads or tracking policies. The only data I collect is the default analytics to give me an idea of how many people are actually interested in what I'm writing, and which pages are getting the most love.

If you'd like to see my thoughts in a more free-form format, with almost no structure, feel free to check out a Mastodon instance I help maintain over at https://linuxrocks.online where I'm @architect, would love to have a larger community of technology and open source enthusiasts.

Defining Project Goals

So while it's still going to take a considerable amount of time, I figured I'd set up a concrete base for what I intend to do in the upcoming months to years depending on how smoothly things progress.

Guides:

So, especially since I've had some fairly good feedback on linux4noobs.reddit.com regarding the general install guide, I'll be refining it as well as covering a few bases that were left out previously. This will include, but is not limited to: the process for installing advanced, manual distros such as Arch and Gentoo, covering what to expect with different types of hardware as well as what to do if the drivers are missing, and some general tips on how to set up your distro after install for use.
On top of that, I'll be looking at guides for KVM/QEMU usage, encrypted drives, gpg keys and other not terribly well covered topics that some users may need help with.

Projects:

The first project I started, aside from this site, is the Forge, which is my personal github repo where currently it just has practice programs. These will of course grow larger and more complex over time, and it holds any important configuration files that I'll be taking with me from workstation to workstation.

Next, there's the LFS builds, currently I'm working on setting up my workstations, but when they're ready I'll be working on 4 different builds of Linux From Scratch, and I'll detail the progress on this site so others can learn along with me. The four builds are 2 Destkop, with and without systemd, and 2 Laptop builds again with and without systemd, the laptop in question is likely going to be my MacBook Air which is currently running Gentoo installed from stage 1 with full volume encryption, lvm partitioning, and a gpg encrypted keyfile, all of which I'll include in my install guides.

Further, I'll be looking at building a custom GNU/Linux distro after getting some experience with LFS. The main goals of my distro will be encryption, user freedom, and security, most likely source-based. I believe it will end up being similar to Gentoo, and really doubt it'd become much of a community supported distro, possibly only being a fun project, but something I'd like to do nonetheless.

After that, I'd like to look at building a custom ROM for my Nexus6 based off the very same distro mentioned above. Which I'm sure will have its own host of new challenges, but it should be a fun learning experience.

Then I'd like to look at building another website, but with a different focus, basically being a huge online diary with TNO encryption, that can serve of an autobiography of the world of sorts. I haven't really worked out the details, but it's something I'd love to see happen.

I'd also like to look at building a randomly generated world with NPCs with randomly generated quests, and no real level cap or endgame. Basically a huge world, similar to minecraft/terarria, where you get to do whatever you want to do. The only real limit would be your ability to control your character and defeat ever stronger beings that aren't always a good idea to fight. Obviously this sort of game would be incredibly difficult to pull off, but by the time I get around to starting it, I'm sure I'll have a much better grasp on what's necessary to make it work.

Beyond even that, and the last project I have in mind is making a completely open smart phone, similar in concept to the Neo900 or the new BlackBerry Priv, but with either my own custom user privacy and security focused ROM installed or no default OS so you can get whatever ROM you want on there. Ideally with a physical keyboard, large battery, connection ports, and waterproofing. Essentially the best possible mobile workstation for programmers, business people andlike-minded people working on large projects they'd like to work on remotely.

On top of all of that, I'm thinking of looking at setting up a method to accept donations in case someone would like to help fund my dreams, with school and learning materials necessary, this could be an expensive undertaking, but I'm determined to get it done one way or another!

Dealing With Power Drain on GNU/Linux for Intel-based machines

So you've just installed GNU/Linux and it turns out things are running smooth, but the battery doesn't seem to last as long, no one told you about this! You expected better performance, not worse! Well, fear not, it's not a problem with GNU/Linux, it's just a problem with a very specific driver. This problem only appears to be effecting intel-based machines, which is odd as intel typically has phenomenal support with GNU/Linux, and honestly, it probably only really stands out to people using thin, portable laptops without a heavy focus on gaming.

So what's to blame? intel_pstate, that's what. This driver is in charge of determining which speeds your CPU can run at, as well as when to pick up the pace and when to slow down. The problem with intel_pstate is that for the past seven months or more at the time of writing this article (2015.07.23), almost all intel CPUs run hot and are being redlined by the frequency scaling driver. Initially I was informed that intel cpus are intended to idle at high frequencies to allow power quickly when needed. This may be true in Windows and OS X, but it's not working as intended in GNU/Linux. Following is how we go about fixing it.

First, we need to disable the intel_pstate driver at boot. The best method to accomplish this is by adding

intel_pstate=disable

to your kernel arguments. The exact location to add this varies by bootloader.

Gummiboot: 

EDITOR=vim sudo -e /boot/loader/entries/$ENTRY.conf
optionsroot=/dev/sdxy rw quiet intel_pstate=disable

Grub:

EDITOR=vim sudo -e /etc/default/grub
GRUB_CMDLINE_LINUX= intel_pstate=disable

Syslinux:

EDITOR=vim sudo -e /boot/syslinux/syslinux.cfg
APPEND intel_pstate=disable

Managing Frequency:

Now that intel_pstate's been disabled, the system should fall back to acpi-cpufreq as the frequency scaling driver, usually with the "ondemand" or "userspace" frequency governors which tends to run the CPU at about 50% of the max speed, so you should see a fair jump in battery life just from that step, though you can take it a bit further with other utilities, like laptop-mode-tools (or tlp, but you can't run both), cpupower, and acpid.

 

ACPID:

This daemon basically allows for management of your system through specific acpi events, like closing the laptop lid, hitting the power button, or reaching a specific battery percentage, which can be useful for a variety of reasons, but the main use I had for it was setting up suspend when I close my laptop, fortunately that works out of the box. It also has the acpi command which gives you battery estimations in the command line.

Laptop-Mode-Tools:

This allows for unified configuration files to manage your system and try to get the battery life you deserve. It does have a configuration file for intel_pstate which may help mitigate the problem with only this tool, but I've found that the acpi-cpufreq driver allows for more control over CPU performance. Once installed, setting the following variables will set your CPU frequencies.

EDITOR=vim sudo -e /etc/laptop-mode/conf.d/intel_pstate.conf
CONTROL_INTEL_PSTATE= 1

EDITOR=vim sudo -e /etc/laptop-mode/conf.d/cpufreq.conf
CONTROL_CPU_FREQUENCY= 1
BATT_CPU_GOVERNOR= powersave

CPUPOWER:

This tool allows you to see your cpu frequency, frequency scaling driver, frequency governor, and even set those parameters manually.

cpupower frequency-info # shows current CPU info
sudo cpupower frequency-set -g powersave # sets the frequency governor to powersave

 

Those are really the only steps I've been able to take to get my battery life back under control on my MacBook Air, prior to this driver, or at least driver update, I could get 12+ hours of battery life without issue, and my MacBook was almost always cold to the touch, after the update, I was lucky to get 2 hours and it was always hot, with this workaround it gets around 6 hours of battery life. Hope it helps anyone in a similar situation.

Edit on 2016.01.03: modifying the commands to a safer method of editing system configuration files, assuming the user is currently using BASH as their shell. If you happen to prefer something like tcsh, like myself, you'll need to run the following command to specify your preferred editor, I happen to like vim. Thank you to /u/moonarch and /u/Trout_Tickler for bringing this to my attention.

setenv EDITOR vim

 

This issue with the intel_pstate cpu frequency scaling driver seems to be resolved as of kernel version 4.5. I only have access to Haswell based computers, but I've heard that Broadwell is experiencing similar issues with the driver, without access to a Broadwell machine, I won't be able to provide any accurate information regarding battery life and efficiency, but the steps involving cpupower, acpid, and laptop-mode-tools should help extend battery life if you run into this issue on any platform later than Haswell.