Crap Nullpointer Exceptions

This is the blog of John Dulaney, a hacker of Fedora, SCAdian, player of Music, blacksmith, sailor, and consumer of Bacon.

Tuning Fedora as a hypervisor

For those of you that use Fedora to host virtual machines, especially if you are using kvm and libvirt, here are some easy tunings to put into place that will improve virtual machine performance.

vm.dirty_background_ratio = 3
vm.dirty_ratio = 15
vm.dirty_expire_centisecs = 500
vm.dirty_writeback_centisecs = 100
vm.swappiness = 10
kernel.sched_migration_cost_ns = 5000000

These settings set the maximum percentages of dirty pages in active and total memory, increase the nimbleness of cleaning them out, set your swappiness to minimum, and reduce the chances of cpu migration.  These settings go into /etc/sysctl.conf

For most, these tunings alone will really improve performance.  However, if you have a system with plenty of memory and generally have multiple vms running all the time, you may want to consider putting hugepages into place.  Hugepages improve the efficiency of memory utilization by breaking memory up into larger chunks, or pages, than is the default.  On most x86_64 systems, hugepages are 2048kb in size.

Before you decide to enable hugepages, you will need to make sure that you have plenty of ram.  A vm that is set to use 1024MB requires 576 hugepages, or 1152MB.  Luckily, adding additional ram to vms or running multiple vms causes this utilization to be fairly linear.  You’ll also want to be sure to keep enough free ram available for anything else you’ll be doing on your system; for most use cases, hugepages shouldn’t take up more than half of the system’s ram.

A good general rule of thumb for most x86_64 systems is to figure out the total ram, in kb, for all the vms you’ll want to use hugepages, then divide by the hugepage size (generally 2048 kb) then add thirty or so, rounding up to the nearest hundred (if the nearest hundred requires rounding down, round to the fifty).  This is your starting hugepage number; experimentation may be required to adjust this for your use case.

To set the number of hugepages, once again, edit /etc/sysctl.conf and add the following line:

vm.nr_hugepages = <number calculated above>

Then, you’ll want to reboot your system to make sure that your memory gets cleared and hugepages get reserved.

Finally, you’ll want to edit each of the vms’ xmls with virsh edit <vm name> and include the following:

<memoryBacking>
<hugepages/>
</memoryBacking>

Then, when you start your virtual machine(s), check your hugepage utiliziation by running grep Huge /proc/meminfo

[jdulaney@volsung ~]$ grep Huge /proc/meminfo
AnonHugePages:    120832 kB
HugePages_Total:    3000
HugePages_Free:     2424
HugePages_Rsvd:        0
HugePages_Surp:        0
Hugepagesize:       2048 kB

Don’t use Free Software because it’s better; use it for your own security

  “In retrospect, something like the Snowden leaks had to happen at some point, but it would have been hard to foresee the actuality in 2013. Those leaks have had a significant effect on our community, making it clear that we are under attack, but also that we arguably have the best defense available against institutionalized surveillance. We have always known that closed-source software can act contrary to its users’ interests; now the wider world is learning that lesson as well. Free software, in the end, may have an important role to play in maintaining the freedom of our societies.”

– Jonathan Corbet

Although I use Free Software because it is often technologically superior, there are other reasons, as well.  With Free Software, you have a much higher assurance that you know what is actually going on with your computer.

It is well known that Apple, especially, has back doors into their products; how else are they able to access your system remotely to ‘fix problems’?

Remember Sony installing rootkits on people’s computers to monitor their music listening?

Do you really trust these companies with your secrets?

Then, on top of this, even if these companies are totally benign and altruistic, the fact remains that you have a *known back door into your system*.  Who’s to say that someone couldn’t exploit this for nefarious ends?

Virtualization on ARM with Fedora

I’ve uploaded an image for the ARM Chromebook that gives a running hardware-accelerate virtual machine (1) as well as the checksum (2).  All passwords are fedora

To run the virtual machine, run /home/virt/virt/Fedora-18-vexpress-xfce-armhfp/boot/boot-kvm as root.  It’s a Fedora 18 image.

Notes:  The kernel does not support usb 3 or wifi; it’s a just enough to get virt working kernel, not all the bells and whistles are built in.

Qemu is built from F20 srpms.  The kernel is patched to fix lpae on exynos5.  It uses a u-boot with hyp mode from Virtual Open Systems.

(1) http://jdulaney.fedorapeople.org/chromebook-kvm.img.xz

(2) http://jdulaney.fedorapeople.org/chromebook-kvm.img.xz.sha512sum

Post-Flock-tem

Day 0

Greg DeKoenigsberg picked me up on his way from Durham, as Fayetteville sits on the route.  On arrival at the hotel, I checked in and saw some Fedora shirts in the lobby.  After having a quick round with the mead I’d brought, I piled my junk in my room, and we all hit the Sticky Fingers up the street.

The Europeans decided to crash; seeing as they were pretty jetlagged, I couldn’t blame them.  I, however, went to the other hotel, and saw Kevin, Maria, Xavier, and Toshio.  We spent the evening catching up, eventually being joined by Jon Disnard.  The discussion turned to Fedora (of course) and tended to resolve around the dual topics of the Fedora Board and Fedora Women.  I also learned that my room mate, Nitesh, was among those suffering from Cancelled Flight Syndrome.

Day 1

The first day started with breakfast in the hotel.  I was disappointed by the lack of bacon, grits, and sweet tea.  The hotels were less than a mile from the school where the actual conference was held, so I walked every day.

The conference itself was kicked off by Robyn’s State of Fedora address.  What she said about working on Fedora out of love was true, at least for me.  I love Fedora, and pull all sorts of hours working on Fedora.  The first talk I attended was Kyle McMartin’s talk on porting Linux and the various Gotchas that can result.  One amusing example from one of the other attendees was a bug that caused all email read by qmail to dissapear without a trace into a black hole, but with qmail repoting in its logs that everything was perfect and dandy.  Despite the obviouse security of this arrangement (no one can read emails that are in a black hole), it was still not exactly a wanted feature.

After, I attended Jon Masters’ two ARM talks, then went to lunch with a whole crowd of Fedora folks.  After lunch I went to Toshio’s Python3 hackfest, where I learned a few useful tips on testing code for Python3 readiness.

I finished up the day’s sessions with Cristoph’s Alternative Desktops talk.  He informed me that Fluxbox does not count as a full alternative desktop.

That evening was the first party, during which I talked with Tim Flink some about my replacement for depcheck.  We’d had some confusion on what exactly was going on, but it is now straight.

Day 2

The morning keynote wound up being delayed by technical difficulties (a Debian laptop not doing the external monitor thing).  As a result, I had fifteen minutes to give my first talk, which I had planned to do in fourty five.  I did get to hit the major differences between virt on x86 and virt on ARM.  My second talk went much more smoothly, although I did find an error reporting bug.  Pretty much the rest of the day was spent doing QA-related stuff, starting out with Tim’s overview of where he wanted Taskbot to go.  This was followed by lunch with some of the QA crew with a few added folks, such as our FPL (and a former FPL).  After lunch, we discussed where Taskbot for a couple of hours, throwing out ideas for implimentation.

After the Taskbot hackfest, I attended the ARM hackfest, which, unfortunatelly, wound up being largely a discussion on which Red Hat folks would be getting which boards between three or four Red Hatters, leaving much of the room out.  This rather annoyed me, as I wound up wasting some valuable time (I could have gone with Joseph and Tim to work more on Taskbot).

Day 3

After the keynote on open source fonts, I met up with Tim and Joseph again to talk about QA stuff.  The first actual talk of the day I attended was the Ansible talk.  After lunch was more working on QA stuff, followed with poking around to see what sort of support for the Beaglebone Black exists in the upstream kernel.  The shindig that night was at the aquarium, and I got to see Fort Sumter (where the Late Unpleasantness began) with my own eyes, though at a distance.  I also took some time to check out a replica of the Hunley, the first submarine to ever sink a ship in combat.

Day 4

The biggest happening on the last day was the first-ever QA meeting in meat-space.  The first half was a discussion on ARM going primary from QA’s perspective.  The second was Visible Cloud, which means cloud images will require QA scrutiny.  Unfortuneatelly, I was not able to attend the second half due to my ride leaving.

All in all, it was a decent conference.  My biggest wish was that there had been more time dedicated to hackfests, since that’s Where Things Get Done.  I was not able to meet up with Kyle McMartin as I had hoped, so some llvm and kernel stuff didn’t get done.

Looking forwards to Flock next year!

South East Linux Fest Wrapup

South East Linux Fest was this past weekend in Charlotte.  The drive there was horrific; most of the way from Cameron to Albemarle I was stuck behind this loony that couldn’t decide if he wanted to drive ten over or fifteen under the speed limit.  It was also pouring down rain, and my driver’s side window got stuck down.  I also heard a horrific grinding sound just after I’d got into Albemarle and had passed the aforementioned looser.  A quart of transmission fluid solved that.

Anyway, once I made it to the hotel and checked in, I met up with Chris Lumens, his wife, and Samantha, one of the newer additions to the Anaconda team.  Later, I met up with Justin O’Brien and eventually Deb Nicholson and a few others and hung out for a while.

Friday morning, I attended David Cantrell’s talk on Anaconda’s new UI, followed by a talk I thought would be on virtual networking, but was actually about another hyporvisor/vm manager.  After that, Ben showed up with the booth components and I helped him set it up.  I stayed there until we called it a day around 6 PM.  Dinner was followed by a round of Cards Against Humanity with Spot, Suehle, and a bunch of Slackware folks.  After an hour of attempted sleep, I awoke to hack on deptcheck mk 2, and wrote most of the actual dependency solver before finally getting to sleep at about 5:30 AM.

Saturday morning, I attended the keynote, and then manned the booth for most of the rest of the day.  During that time, I helped a couple of people out with setting up Fedora on the ARM Chromebook and built the 3.10rc5 kernel and gave it a whirl.  We closed up shop a little after 5 PM.

Saturday evening was productive in terms of talking to some Haskell folks about porting ghc to armv8.  The general conclusion was to see how well llvm supported the new arch, since it seems that ghc is going to be using llvm for its back end.

Sunday, I was at the booth from 9 AM to about 1 PM, when things wound down and we packed up.

All in all, a decent SELF.  I was glad to see folks I hadn’t seen in a while, and am looking forwards to next year.

Announcing my new project: Bringing the MIT Whirlwind I to simh

Almost five years ago, my mentor and instructor in how to program passed away from cancer after a long and full life. His software writing started with the MIT Whirlwind I almost sixty years ago. As a tribute to his memory, I have decided to write an emulator for this computer and try to get it included in the simh suite. Right now, I am researching the computer and beginning to formulate my approach to the project.

Wish me luck!

FUDCon NA 2013

This year’s North American Fedora Users and Developer’s Conference (FUDCon) was in Lawrence, Kansas.  I took the train from North Carolina, with layovers in Washington and Chicago.  The trip went very well: I had lunch in DC with Brian Kemp and then in Chicago I played tourist.  All along the way I took photographs of trains (those of you that know me will know why) that are unusual back home.
I arrived in Lawrence around midnight Friday morning; Jon Disnard (masta) picked me up from the train station.
FUDCon itself kicked off with an abbreviated (that always seems to happen) State of Fedora speech by Robyn Bergeron, our esteemed Fedora Project Leader.  In the speech, Robyn pointed out that Fedora remains strong, Fedora 18 is finally out, and that the future looks bright and shiny.  Fedora 19 shall be known as Schrodinger’s Cat; there is currently no set release date due to the massive slippage of Fedora 18.
After Robyn’s talk, Barcamps were proposed and voted on.  I started out with the two ARM barcamps; the first was a general overview of the state of ARM in Fedora, the second was a demonstration of the new 24 core Calxedra ARM based servers that will be used as build machines for Fedora ARM.
After lunch were several lighning talks.  The most interesting for me was the one on pkgwat, which can be used to querie Bodhi on status of various packages; whether they are in testing, stable, how much karma they have, etc.  I was also fairly interested to find out that the GPIO on my Raspberry Pi can be controlled from the command line.  This is something that can be used to teach basic programming.
I went to Spot’s talk on overhauling the Fedora release process model, which turned out to not be quite what I expected.  Spot’s proposal is to go to a release cycle similar to the old Red Hat Linux cycle:  have a major release (x.0) that may not have all the polish and all the major features quite done, but gets the code out there, with the following three releases adding more and more polish and stability.  I commented that I worried that this could either put extra load on the QA team and package maintainers or potentially keep out ultra new features in later minor releases, going against the fourth foundation of “first”.
Afterwards, I attended Tim Flink’s blocker bug process talk so that I could provide input and back him and Adam Williamson up on questions.  The talk quickly became a Q&A/Discussion session, but considering the mix of non-QA people present, this led to some comments on the inadequacy of the current system in a few areas (to be farther discussed in a bit).  I finished up the barcamp session with the talk on Fedora Formulas; a potential Ansible replacement for spins.  The idea is to use Ansible playbooks to customize packages installed and config post installation of Fedora.
Friday evening I went to dinner with the ARM team, and after much revelry returned to the hotel to catch up with Tim for a while prior to turning in.
Saturday was largely spent with the ARM folks.  The first half of the day was discussing various aspects of the ARM project, including push to Primary Arch, dropping support for ARMv5, adding support for v7, and the v8 bootstrap project.  There was also discussion on employing new Calxedra hardware and migrating the builders from Seneca to a Red Hat facility in Pheonix.  During this I explained the blocker bug process and set up blocker trackers for F18 Final for ARM and F19.  More on this in a bit.  That afternoon the focus was on hacking up various fixes for problem packages and some experientation.  Jon Masters also gave a short presentation on the kernel on ARM devices and where things were located, which will be very useful.  The Calxedra guy also allowed access to a few cores on his server, so I grabbed one to play with, setting up an rpm build environment and trying to see if I could build a couple of problem packages.  I was also hacking on my Raspberry Pi, running the configure script in preperation for attempting to build GCC 3.8.  I was also helping Kiara with setting up her Raspberry Pi.  The networking to pull all this off was rather Rube Golberg in nature:  dhcp and access to the school’s wireless was done via the Calxedra guy’s laptop, which was bridging the wireless to its Ehternet port, which was tied into an Ethernet switch.  This switch had a wireless access point attached (which I was tied into with my laptop) as well as Jon Master’s laptop, the Calxedra server, and Kiara’s Raspberry Pi.  I had my own switch hooked to the Ethernet port on my laptop, which was tied to my Raspberry Pi and Kiara’s laptop; her Raspberry Pi was getting power from my laptop over USB.  None of this counts the other folks in the room tied into the wireless AP to gain access to the server.
Saturday evening was the always-awesome FUDPub, which featured a buffet and open bar.  I moved between tables some talking to folks.  After FUDPub many of us went bowling; this year I knew better than to bowl against Adam Williamson.  I managed to hit the high double digits (yay!).  After this we returned to the hotel where there was more socializing, some Youtube watching, and virtual machine building.
Sunday was QA day.  Adam, Tim, and I discussed the current blocker bug naming scheme and how confusing it is.  After some discussion, we came up with a new scheme that Adam has posted to the Test list for comment.  We then started discussing how to manage blocker bugs outside of Bugzilla, but I temporarily left the discussion to explain to Paul Whalen in more detail the release process from QA’s perspective so that he can then better bring the rest of the ARM folks up to speed.  I reccomended that ARM follow this process as closely as possible (with some modifications needed for ARM to remove non-applicable criteria and the like).  He then had to leave for his flight.  I rejoined the discussion on the new blocker bug proposal web app until Tim pinged Emily Dirsh to help him with UI design.  Adam and I then finished up some loose threads with the earlier blocker tracking name scheme discussion.  Adam then went off to join Tim and I met up with lmr to create a virtual machine with an autotest server to familiarize myself with autotest as it stands now.  After getting that set up, I rejoined Tim and Adam for the end of the blocker app discussion before we retired to the hotel.
This was an excellent FUDCon; I did get quite a lot of work done and learned a few things that will be useful going into the future.  It was also good to catch up with people again, many of whom I had not seen in a year.  I am very glad to have been able to go and hope to make it next year.

FUDCon wrapup

So, I like wraps.  Beefy miracles are also good (and I got some Beefy Miracle on my whit FUDCon shirt Saturday).  Solution?  Put Beefy Miracles in wraps.

Anyway, Saturday was Barcamp Day.  After the group photo, barcamp proposals/voting was Jared’s speech.  His message was essentially that Fedora is strong and that we need to keep going.  My first barcamp of the day was on Boxgrinder.  Boxgrinder makes automated VM creation easy, something useful for upcoming AutoQA tasks.  During lunch, I had the aforementioned Beefy Mishap.  Afterwards, I went to the Fedora ARM barcamp.  I’ll be honest, it was not entirely motivated by my desire to learn more about Fedora on ARM; they were giving away hardware.  Those of us that wanted some lined up and gave pitches as to why we should get it.  Mine was simple:  I said, “I’m QA.”  Besides Adam Williamson firing me a couple of times, things went well.  I got a few very basic ideas as to where the ARM project is going and what sort of things us QA folks will need to be doing.

After that, I went to see what the Sys Admin Study Group was up to, and sat in there for an hour and hacked.  After that was the State of the Fedora Kernel address.  As I had fairly gathered, there really isn’t anything earth shattering that has taken place in the kernel recently besides the version change.  I wish the clustering barcamp had been the hour before, since I’d have liked to have attended both.  Alas, I’ll ping the barcamp owner.  The last session was Sandro’s QA session.  I went to help him out with that, and things went well, with lots of questions asked and the process explained rather well.

Fudpub was awesome.  One dude I bowled against bowled a higher score than everyone else on the lane combined, but it was still fun.  The food was good, even if the beer selection was a bit lacking.

After Fudpub, I went through a couple of Poker games.  The first I won, the second I lost, twice.  After the second loss, it was time to go horizontal.

Sunday was spent with the Anaconda team’s hackfest.  We hashed out some ideas for testing the new UI and I put in my two cents here and there.  If there is one tiny thing about the new UI you don’t like, that’s probably a result of my input, so blame me.

Sunday afternoon I rode with Will Cohen and Greg DeKoenigsberg.  We took a back road for the first forty miles.  Winding through the mountains of western Virginia with snow on the ground was an awesome sight.  We arrived in Raleigh early enough for me to catch the 7:00 Greyhound back to Fayetteville so that I wouldn’t have to sit around until 11 PM.

Overall, I’m glad I went.  I got a fair bit of work done and learned a lot that is pertinent to my future goals within the Project.  It was nice meeting up with folks I already knew and meeting others face-to-face for the first time.

First Day of FUDCon

First day of FUDCon is pretty close to over.   Today started out rather slow; I didn’t get down to breakfast until around 8:30.  I sat with Christoph Wickert, Adam Williamson, Peter Robinson, Tim Flink, and someone else who’s escaping me at the moment.

I kicked things off with the Infrastructure hackfest.  After two hours of that, it was time for lunch, although I didn’t eat anything.  This afternoon, I went to Paul Frield’s Drupal workshop, followed by figuring out where to start working on AutoQA self testing.  After that, I failed a test.  Since then I’ve been sitting in front of the fire place drinking Tatica’s wonderful rum, talking, and hacking AutoQA.

I’m looking forward to tomorrow, for sure.

FUDCon Blacksburg 2012 is nearly here!

Attention, people of Earth.  This is Prostetnic Vogon Jeltz …

Oh, wait.

Anyhow, in a couple of days I’ll start my great Fedora quest.  Hopefully I’ll roll a natural twenty and won’t have any issues in travel.  I will be riding with the amazing Andy Grimm and the great Will Cohen.

I plan to attend Paul Frield’s Drupal talk Friday as well as hosting an AutoQA hackfest.  The rest of Friday I plan to attend various hackfests and maybe do something to cloud things up.

Saturday will probably be very cloudy, as I can use all the knowledge I can get in that area.  It looks like there will be a bit of a QA barcamp at some point, and I may show up to that to mommick Sandro some.  Sunday will be largely taken by more QA stuff, especially the hackfest where we’ll be mucking about figuring out the best way to test the new Anaconda.

Well, that’s it.  I’ll let every go back to their visit from the Vogons.

John.

Follow

Get every new post delivered to your Inbox.

Join 50 other followers