Wednesday, March 22, 2017

podcast: advantages of meditation

lots of great ideas and info in this podcast on the advantages of meditation. Sam Harris interviews Joseph Goldstein.

Tuesday, March 21, 2017

new short talk: Distributed Processing

My new talk is about different processing tradeoffs, using good old Unix utility xargs, Python multiprocessing, gevent, and some distributed processing tips.  Also: tonight there's a seekrit announcement -- watch this space, and/or follow me at @JohnTellsAll

Talk URL is http://bit.ly/jta-dist (this page, which will receive updates)

Slides are in Google Docs

Sunday, January 22, 2017

Favorite podcasts

I listen to a lot of podcasts. Some are moderately heavy and technical, some are lighter stories and just for fun. Many encourage me to be a better person, or help me to move forward to achieve my goals.

Here are a few favorites:

- 99% Invisible: this podcast is fun and interesting!  Their specialty is "intriguing design of everyday things", often architecture. Recently they had an episode of mini-stories, one on the design of the standard ("Snelling") eye chart. It turns out everyone uses it because it's easy, but not because it's the best. It's easier for the doctors, but not necessarily for the patients, it's not as accurate.
Another mini-story was about "copyright traps" in maps. Map makers spend a great deal of time and effort on their work, so they insert fake towns into their maps.  If another map copies their work, they'll copy the fake town, so they'll get busted for copyright violation. In one case people set up a few businesses and homes in a new area. They looked up on the map what they should name their town, they saw a name, so they used it.  A "fake town" became a real one!

- New Stack Makers: As an "expert on quality dev and devops"*, I keep up to speed with what people are doing in terms of web server systems. The trend in the last few years is for containers, specifically Docker containers, and also microservices.  These tools give us power and flexibility at the expense of making things explosively more complex. This podcast helps me stay up-to-date on what practical steps people are making to create, manage, update, and simplify their server infrastructure.

- Entre Leadership Podcast: Again as an expert, commonly I find myself in a leadership position. Generally I'm the most senior person, so people look to me for guidance and perspective. This is a challenge for my enlightenment and humility, and I take this role seriously. By listening to this and other leadership podcasts, I learn how to side step my ego and help people come up with their own plans, execute them, and get feedback.


Tuesday, December 20, 2016

Using multiple worktrees with git

Modern development with Git is great. It's really easy to work on a feature branch, make changes, undo changes (oops), and share work with colleagues. However, there are challenges. If I'm running tests on my local "beer" branch, I can't easily switch over to the "gin" branch to edit code. The local environment only has one feature branch checked out at a time. I can run tests, or add code, or switch branches, but I can't do two things on two different branches at once...

But now I can! Thanks to Manoj Mahalingam S, I know understand the "git worktrees" feature which lets you easily use multiple branches at the same time. Thanks, Manoj!

Using multiple worktrees with git

Friday, November 4, 2016

TIP: recover file from old Git branch

In cleaning up my source code I realize I was too... aggresive in my house cleaning. I deleted a file I needed.  No biggie, just take a look at HEAD or HEAD^... Hey, that didn't work!

TIP: use git rev-list to search and find the most recent commit that affected (deleted) a path. (Source: Charles Bailey on SO)

$ git rev-list -n 1 HEAD -- docker/torta/torta-task.json
0c400d323aff484cfb2bdc18dcdb7813de93a658

Yay!  That was really easy. Now all I need to do is check out from that Git hash and I'm done:

$ git checkout 0c400d323aff484cfb2bdc18dcdb7813de93a658 !$
error: pathspec 'docker/torta/torta-task.json' did not match any file(s) known to git.

Err?  Hmm. Oh: the 0c4 branch contains the "delete" command. If I want to get the file contents, I'll get it from the branch just before that one:

$ git checkout 0c400d323aff484cfb2bdc18dcdb7813de93a658^ docker/torta/torta-task.json
$ ls -l docker/torta/torta-task.json
-rw-r--r--  1 johnm  staff  1445 Nov  4 13:06 docker/torta/torta-task.json

Yay, it worked!  Thanks, interwebs!

Tuesday, November 1, 2016

Videos! For "Platonic Solids of Quality" and "Functional Programming and Django QuerySets"

I've been remiss in posting my videos.  The last two talks went really well

- Platonic Solids of Quality

As people who produce awesome stuff, there are a lot of tools we can use to work less and simplify our workflow. This talk introduces many techniques from Agile Testing and other sources so we can all do more things faster.

- Functional Programming and Django QuerySets

This talk is a quick overview of Python iterators and why to use them, then combining iterators in Functional Programming (vs procedural and object oriented programming). Lastly I show how to use these ideas in order to better understand and test Django QuerySets.

I have another talk video somewhere, but it's about using Selenium for testing and thus has rather a bit of profanity in it.  No link.

Thanks to Don Westland for recording-editing these for me!

Friday, October 28, 2016

TIP: use Strace to debug issues inside Docker

Yesterday my Docker application wasn't working correctly -- the appserver is hanging.  Debugging this is a challenge: there's no crash nor stack trace to point out the issue.  Is the appserver misconfigured, so it's trying to talk to a non-existent database? Is the config okay, but the network is not set up correctly?  Can an external service not see our Docker container correctly?

To debug this I used my good old "strace" command to trace exactly what is happening. It outputs log messages for all system calls the appserver does, including all the network I/O.  Alas it didn't work for me:

strace: test_ptrace_setoptions_for_all: PTRACE_TRACEME doesn't work: Operation not permitted

This is odd, as the Docker container is running with root permissions, and the parent container is Debian.

My buddy Loren says this is a Docker thing -- the ptrace system call (which strace uses) is disabled by default.  To run a Docker container, re-enabling ptrace, run this:

docker run -i -t --security-opt=seccomp:unconfined --rm debian sh -c 'apt update ; apt install -y strace; strace -e trace=network ping -c1 8.8.8.8'

The above command does some Docker stuff, then sends a single ICMP ping packet to Google's global (and easily-remembered) DNS server.

Output:

socket(PF_INET, SOCK_RAW, IPPROTO_ICMP) = 3
setsockopt(3, SOL_SOCKET, SO_BROADCAST, [-68864700], 4) = 0
PING 8.8.8.8 (8.8.8.8): 56 data bytes
sendto(3, "\10\0\363\330\0008\0\0b\207\23X\0\0\0\0\4~\f\0\0\0\0\0\0\1\2\3\4\5\6\7"..., 64, 0, {sa_family=AF_INET, sin_port=htons(0), sin_addr=inet_addr("8.8.8.8")}, 16) = 64
recvfrom(3, "E\0\0TU\r\0\0%\1\204y\10\10\10\10\254\21\0\2\0\0\373\330\0008\0\0b\207\23X"..., 136, 0, {sa_family=AF_INET, sin_port=htons(0), sin_addr=inet_addr("8.8.8.8")}, [16]) = 84
64 bytes from 8.8.8.8: icmp_seq=0 ttl=37 time=0.339 ms
--- 8.8.8.8 ping statistics ---
1 packets transmitted, 1 packets received, 0% packet loss
round-trip min/avg/max/stddev = 0.339/0.339/0.339/0.000 ms
+++ exited with 0 +++

Note the "sendto" and "recvfrom" system calls.  They show that the Docker container can talk to the internet.  This is verified with the "packets" line.

Strace is so awesome, Julia "b0rk" Evans wrote an entire zine about it! ~ http://jvns.ca/zines/

Thanks Loren and Julia!