Thursday, September 25, 2014

Happy-ness

I've been thinking about happiness lately, and some of this has to do with the current sermon series at church—not that my thoughts have been all that spiritual. Many have commented that happiness can't be found by directly pursuing it; rather, happiness comes when we're pursuing something else.

I was thinking about this last night. I had paused from whatever I was doing at the computer (it was so important that I don't even remember what it was) to read to Sheri. I read a sentence or two from The New Machine Age but without context it didn't seem all that good a fit. I switched to The Wisdom of the Sadhu, which worked better, though one of the parables didn't really work for her.

After a while I went to the kitchen, remarking that the dog probably wanted her teeth brushed. I opened the cabinet where her dental supplies are kept, and heard her jump from the couch. Her toenails clicked on the hardwood floor as she sped toward the kitchen, and by the time I had the toothpaste on her toothbrush, her forepaws were on my leg as she expressed her joy.

Her joy was contagious, and I remembered hearing that people live longer if they have someone to care for. Come to think of it, I saw a bumper sticker recently: on one side was a large stylized paw-print, and the words were: "Who rescued who?" or maybe "whom"…

Saturday, September 20, 2014

Dishes weren't getting clean in Kenmore Elite dishwasher 665.13422K701

So I noticed that the flatware wasn't getting clean in the dishwasher, and the cups weren't looking so nice either. Things were wet, mostly, but…

I looked at the dishwasher manual for troubleshooting help; no joy. So I did some web searching, and followed their advice (mainly, check for food junk which might impede water flow). The good news was we didn't have food junk impeding the water flow.

So I did some guerilla diagnostics, by which I determined that water was spraying from where it shouldn't (blech). And that the spinning sprayer arms didn't. After getting the kitchen floor quite wet, I noticed a little lever assembly near the axis of the lower spinning sprayer arm; it's in the photo at left. (There's probably a better word for "lever assembly" but I hope you get the idea.)

I zoomed out a bit to take the image at right, which shows the lever in context.

What I noticed was a lot of water shooting out from the gray plastic thingie directly beneath the lever assembly. A plastic ell enters the "floor" of the dishwasher right near there. The lever ought to keep the ell in the floor. When the lever is out of position, water pressure can force the ell out of its position in the floor, and water sprays from where it shouldn't. This reduces water pressure in the (non)-spinning (non)-sprayer arm, resulting in a poor wash.


The next three photos at left show the lever returning to its proper position. Notice the catch, or bracket, that keeps the lever from moving up (vertically).

With the lever locked in place, it holds the ell in its proper position, and the water seems to go where it should, spinning the sprayer arm and giving us clean dishes once again.

I was going to ask why they didn't put that in the manual, but I suspect this lever doesn't get moved very often. Actually, I don't even know how ours got thrown out of whack, but it definitely was.

Network issues solved: replaced D-link DIR-810L by ASUS RT-N66U

Streaming video to our computers, or even audio, was problematic at our house. It had been some months (don't remember how many months exactly), but we had these other issues too:
  • ssh to my ISP's shell server, and after a few minutes, response would become glacial. I mean, type "ls" and it's literally minutes. If I did it right after login, it was fine; it was only after a few minutes it would act constipated.
  • downloads of any size took "forever". We tried to download a map upgrade so the lovely Carol could update her GPS; eventually I wrote a script to kill the download and restart it (fortunately it could pick up from where it left off). It wasn't as straightforward as "wget -c", because we started the download by clicking on some icon, then hitting some button and… (I hate computers.)

    Fortunately, by saying "ps wwx" or something like this I was able to get the program's parameters. The script I wrote involved monitoring the most recently updated file in the most recently updated directory (many files were getting downloaded, with a new directory getting created every so often), then killing and restarting the download process when it looked like things had paused for more than a minute or two.

We have DSL and are supposed to be able to get something like 5-6 Mbits per second. And we often have; we watch streamed video on netflix without problems. So that puzzled me.

On another occasion, Sheri wanted to download a 200-Mbyte package; it never finished. I thought it was the wireless, so I ran a long cable to her computer—not much better. I got the URL and just said "wget" on my own computer, something like this:

collin@p3:/mnt/home/collin> wget -O frsxprodmg https://dl.dropboxusercontent.com/blahblah/FRSXpro3.5.6installer.dmg
--2014-08-26 21:58:03--  https://dl.dropboxusercontent.com/blahblah/FRSXpro3.5.6installer.dmg
Resolving dl.dropboxusercontent.com (dl.dropboxusercontent.com)... 23.21.100.40, 107.22.226.183, 184.73.171.70, ...
Connecting to dl.dropboxusercontent.com (dl.dropboxusercontent.com)|23.21.100.40|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 218103808 (208M) [application/x-apple-diskimage]
Saving to: `frsxprodmg'

21% [=======>                               ] 47,504,787   624K/s  eta 4m 36s  ^C^Cc^C^C^C^Z
It sat there, paused, for quite a while. Then, as you can see, I got frustrated and hit some keys to stop it. I then restarted.
collin@p3:/mnt/home/collin> 
collin@p3:/mnt/home/collin> wget -c -O frsxprodmg https://dl.dropboxusercontent.com/blahblah/FRSXpro3.5.6installer.dmg
--2014-08-26 22:00:18--  https://dl.dropboxusercontent.com/blahblah/FRSXpro3.5.6installer.dmg
Resolving dl.dropboxusercontent.com (dl.dropboxusercontent.com)... 54.225.142.207, 54.221.243.250, 54.225.177.135, ...
Connecting to dl.dropboxusercontent.com (dl.dropboxusercontent.com)|54.225.142.207|:443... connected.
HTTP request sent, awaiting response... 206 PARTIAL CONTENT
Length: 218103808 (208M), 170541677 (163M) remaining [application/x-apple-diskimage]
Saving to: `frsxprodmg'

41% [++++++++=======>                       ] 90,094,310   393K/s  eta 4m 37s  ^C
It hung after another, what, 43 million bytes? I restarted it a few more times, getting varying amounts of data in before the download hung. I was very thankful for "wget -c" but even with that, the experience wasn't pleasant. 18MB, 65MB, 2MB, then about 43MB to finish.

Now I will pause here to remember that we live in an astounding age, where in less than half an hour, we can download something like 200 Mbytes of software and be quite confident that the data were not corrupted during transmission. That said, with all this technology it's supposed to be easier.

It was about at this time that I remembered the issues I'd been having with my ssh (login) sessions to my ISP. Could all this be caused by the router? I asked my ISP's support folks.

They confirmed that yes, this behavior could be caused by a flaky router. In my case it was a D-link DIR-810L, which we've had since October.

They suggested I try without the router. Ooooh, OK, so I logged in as root and unmounted all the NFS. Turned off as many services (xinetd, sshd, etc.) as I could figure out easily, changed the network configuration for DHCP, and connected it directly to the DSL modem (disconnecting the rest of the home network).

But I couldn't connect to anything--had to route add default gw with an "educated" (yeah right) guess. I hate computers.

Once I figured that out, I ran wget and… you guessed it—the entire 207MB+ came down in one shot, about five minutes So it was the old router!

After asking some folks at work, I went shopping. The recommended Buffalo item wasn't available locally, but I was looking for the ASUS RT-N66U dual-band wireless-N900 router. Best Buy (near Home Depot!) had an RT-N66R, but I didn't know what the difference was between 'R' and 'U' so was skittish.

I ended up ordering it from amazon and using "amazon locker" for delivery. This worked quite well. The idea is, rather than having to run over to the UPS office ("What are your hours again?"), I can have it delivered to a nearby 7-11 (open 24 hours a day). Amazon.com emails me a secret code, which I key into the locker thingie, and one of the dozens of doors pops open; inside is my item.

So I got it home and hooked it up. Configuration was a breeze, even though I have a somewhat unusual configuration (default gateway is 192.168.1.254, not .1.1). The router automagically downloaded a firmware upgrade (this operation went quite quickly, which made me feel confident in my purchase). I then tried the 207MB download, and it came in under five minutes. Whee!

Icing on the cake: logins to the server don't go all molasses-in-January-in-Minnesota on me after a few minutes.

One more thing: I naïvely set the SSIDs to the same thing on both the 5GHz and 2.4GHz wireless networks. This seemed to confuse the macbook air's wireless. (Was it cycling between the two networks, which btw also shared the same password?) After I separated the SSIDs, the problem went away.

Friday, September 12, 2014

42% “efficient”

Suppose a team of coders work on a project, and we say we want code reviews, or maybe inspections. We have estimates for how long various parts will take to code -- adding up to 3 engineer-weeks, say. How much effort (engineer-days) should we allocate to cover code reviews, discussion and rework?

Well, that depends of course on how much time we think it will take to review and discuss and rework the code. Let's suppose for a moment that if my colleague spends a week coding and doing some basic testing, then I will stare at the code for two days. Actually, let's say there are two reviewers, so two of us will stare at the code for two days each.

Then we have a day where three of us meet (in the morning, say) and talk about the code for a while. After 1½ or 2 hours, we go back to our desks and think some more about what we heard, and the person who wrote the code makes some changes. We repeat that but at the end of the day we're all happy with the code.

As I write this, that seems like a lot of review time, but if we do it that way, the total engineer-days spent add up thus:

  • 5 engr-days coding
  • 4 engr-days reviewing (2 reviewers @ 2 days each)
  • 3 engr-days discussion and rework (3 participants @ 1 day each)
Twelve (12) days total, for a thing whose coding estimate was 5 days.

That works out to 5/12 ≈ 42%. In other words, if you have 10 engineer-days available, then at the above rates of "overhead" you can spend only 4.2 days coding; the rest is review, discussion, rework. That doesn't include integration or system test, etc.

Perhaps I have the wrong ratio of (review + discussion + rework) to (code and basic test), but I'll confess to being surprised at the result of 42% given the above.

Monday, August 11, 2014

One size fits… one.

Today's Old Testament reading, from the book of Nehemiah, shows an interesting contrast in the way different people experience God. In chapter 1 of his book, Nehemiah tells us that he was cupbearer (or wine-taster) for the king in Babylon. He heard about the condition of Jerusalem and was very upset that the walls were broken down and the gates destroyed.

In chapter 2, he asks the king to send him “to the city in Judah where my fathers are buried so that I can rebuild it.” He doesn't mention the name Jerusalem (there probably was some sensitivity about that particular city), but the part I found particularly interesting is what came next:

I also said to him, “If it pleases the king, may I have letters to the governors of Trans-Euphrates, so that they will provide me safe-conduct until I arrive in Judah? And may I have a letter to Asaph, keeper of the king's forest, so he will give me timber to make beams for the gates of the citadel by the temple and for the city wall and for the residence I will occupy?”

And because the gracious hand of my God was upon me, the king granted my requests. So I went to the governors of Trans-Euphrates and gave them the king's letters. The king had also sent army officers and cavalry with me.

Nehemiah 2:7-9
What's so interesting about that? Well, maybe not so much in itself, but what I noticed was the contrast with Ezra, which was in the readings from some days past. The part of Ezra I'm thinking of was this:
There, by the Ahava Canal, I proclaimed a fast, so that we might humble ourselves before our God and ask him for a safe journey for us and our children, with all our possessions. I was ashamed to ask the king for soldiers and horsemen to protect us from enemies on the road, because we had told the king, "The gracious hand of our God is on everyone who looks to him, but his great anger is against all who forsake him."

So we fasted and petitioned our God about this, and he answered our prayer.

Ezra 8:21-23
What I take from these two men of God is not that Ezra trusted God and Nehemiah trusted in military escorts. And it's not that Nehemiah was bold and Ezra was timid. What I get is that God deals with us in a way that's suited to us. He knows us individually, knows our personalities and inclinations and strengths and weaknesses (besides our joys and tears and aspirations and disappointments).

He does not treat us like interchangeable parts; he knows us and cares for each one of us. And the way we honor him is something that's unique to each one of us too. How do I love my children? I love each one for who she is. And that gives me a little picture, a little sense of appreciation for how God loves me—he loves me in a way that's different from the way he loves you, and our paths will be different, and that's okay; indeed, it is very good.

Thursday, June 26, 2014

Disable new user signups (registration spam) on a wordpress blog

The lovely Carol has a wordpress blog. She's been getting lots of "new user registrations," and she spends too much time deleting these new "users". These are not real people; they're more likely "spammers wanting to take control of the system through multiple user registrations" according to this site.

I did a web search on "wordpress new users" or something like this (no quotation marks) and found an article titled “How to Disable New User Registration on WordPress Using PhpMyAdmin,” which was useful, except I don't have PhpMyAdmin. And I don't have cPanel either. And that was the most promising of the links that came up. Bummer.

What's a geek to do? well, this:

$ ls
dec24.html                 wp-config-sample.php
fire-safe-box/             wp-content/
INDEX.HTML.oneclickbackup  wp-cron.php
index.php                  wp-includes/
license.txt                wp-links-opml.php
md5sums.txt                wp-load.php
readme.html                wp-login.php
wp-activate.php            wp-mail.php
wp-admin/                  wp-settings.php
wp-blog-header.php         wp-signup.php
wp-comments-post.php       wp-trackback.php
wp-config.php              xmlrpc.php
$ 
H'm, wp-signup.php you say? That looked promising. I noticed this:
 30 if ( !is_multisite() ) {
 31         wp_redirect( site_url('wp-login.php?action=register') );
 32         die();
 33 }
"wp-signup.php" [readonly]
THAT looked promising. I looked in wp-login.php and searched for "register"; I found this:
646         if ( !get_option('users_can_register') ) {
647                 wp_redirect( site_url('wp-login.php?registration=disabled') );
648                 exit();
649         }
"wp-login.php" [readonly] 921 lines
O-kay! So we want to kill off the option 'users_can_register'.

By looking around, I discovered that get_option() gets things from the cache. I searched wordpress.org for "get_option" and landed at http://wordpress.org/search/get_option, which refers to the options database table. Somehow I got the idea of looking in wp-config.php where I discovered this:

// ** MySQL settings ** //
define('DB_NAME', 'bedrockdb');    // The name of the database
define('DB_USER', 'fred_flint');     // Your MySQL username
define('DB_PASSWORD', 'yabbadabbadoo'); // ...and password
define('DB_HOST', 'hanna.bar.be.ra');    // 99% chance you won't need to change this value
(Of course the names are changed to protect… well, myself.) In another window, I typed:
$ mysql
ERROR 2002 (HY000): Can't connect to local MySQL server through socket '/tmp/mysql.sock' (111)
D'oh! I had to fill in the hostname and the username and…
$ mysql -h 'hanna.bar.be.ra' -u 'fred_flint' -p
Enter password: <here I typed “yabbadabbadoo”>
Welcome to the MySQL monitor.  Commands end with ; or \g.
Your MySQL connection id is 24209555 to server version: 5.5.37-35.1-log

Type 'help;' or '\h' for help. Type '\c' to clear the buffer.

mysql> use bedrockdb
Database changed
mysql> show tables;
+-----------------------+
| Tables_in_bedrockdb   |
+-----------------------+
| wp_commentmeta        |
| wp_comments           |
| wp_links              |
| wp_options            |
| wp_postmeta           |
| wp_posts              |
| wp_subscribe2         |
| wp_term_relationships |
| wp_term_taxonomy      |
| wp_terms              |
| wp_usermeta           |
| wp_users              |
+-----------------------+
12 rows in set (0.02 sec) 
Okay, that's good. Now which table? wp_options? Yes! And according to the above article, I want wp_options. So I said:
mysql> describe wp_options;
+--------------+---------------------+------+-----+---------+----------------+
| Field        | Type                | Null | Key | Default | Extra          |
+--------------+---------------------+------+-----+---------+----------------+
| option_id    | bigint(20) unsigned | NO   | PRI | NULL    | auto_increment |
| option_name  | varchar(64)         | NO   | UNI |         |                |
| option_value | longtext            | NO   |     | NULL    |                |
| autoload     | varchar(20)         | NO   |     | yes     |                |
+--------------+---------------------+------+-----+---------+----------------+
4 rows in set (0.01 sec)

mysql> select * from wp_options where option_name = 'users_can_register'; 
+-----------+--------------------+--------------+----------+
| option_id | option_name        | option_value | autoload |
+-----------+--------------------+--------------+----------+
|         6 | users_can_register | 1            | yes      |
+-----------+--------------------+--------------+----------+
1 row in set (0.02 sec)
It's been a while since I updated a table in MySQL. Naturally I went to a web search to find a list of SQL commands. This one told me what I needed, down in "Data Commands."

Let's see, I wanted to change the option_value from 1 to 0 where… OK, I tried this.

mysql> update wp_options set option_value = 0 where option_name = 'users_can_register';
Query OK, 1 row affected (0.01 sec)
Rows matched: 1  Changed: 1  Warnings: 0
Wow, it worked the first time! How often does that happen? Just to be sure, I typed this:
mysql> select * from wp_options where option_name = 'users_can_register';
+-----------+--------------------+--------------+----------+
| option_id | option_name        | option_value | autoload |
+-----------+--------------------+--------------+----------+
|         6 | users_can_register | 0            | yes      |
+-----------+--------------------+--------------+----------+
1 row in set (0.01 sec)

mysql> quit
Bye
Now for the acid test. I went to the sign-up page "wp-signup.php" and got: "User registration is currently not allowed."

And that's what I'll do next time.

Thursday, June 12, 2014

CSV to QIF transformation with Python3

This evening I discovered that the folks at my bank, or more probably the folks running the online banking website, have "upgraded" the user interface, and they no longer support downloading transactions to the "old" Quicken file format (i.e., "QIF"). Instead, they (mostly) support "export"ing transactions to CSV.

Fortunately, a format conversion from CSV to QIF isn't that hard. I first did a web search, which yielded

  • a website (upload your csv and we'll give you a qif file),
  • some Windows (I guess) software--free to download and try, but I guess they want money for it, and
  • maybe something else that I didn't think would work for me
so I decided to just hack something together. Of course it would be in Python, and probably python3.

I remembered that there was a module that knew how to read and write CSV files; a quick look at the library reference showed that the module is called, oddly enough, "csv"; the result is described here (produced by "pydoc3 -w csv_to_qif")

 
 
csv_to_qif
index
/mnt/home/collin/projects/csv-qif/csv_to_qif.py

transform a "csv" file on stdin to a "qif" file on stdout.

 
Modules
       
csv
os
sys

 
Functions
       
main(infile, outfile)
Read infile -- we expect valid entries to have 9 fields:
xaction#, date, descr, memo, amt debit, amt credit, bal, check#, fees
 
Write an outfile that looks like...
    !Type:Bank
    C*
    Dmm/dd/yyyy
    T<amount> (negative for withdrawal)
    N###      (## = checkNumber)
    P<descr>  (description...)
    ^
Each entry begins with "C" and ends with "^".
Here's the code. It is Copyright©2014 by me, but you are free to use, redistribute and or modify it under the terms of GPL2; see below.
#!/usr/bin/python3 -utt
# vim:et
'''transform a "csv" file on stdin to a "qif" file on stdout.'''

# This program is free software: you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
# the Free Software Foundation, version 2.
# NO WARRANTY express or implied.

import csv
import os
import sys

def main(infile, outfile):
    '''Read infile -- we expect valid entries to have 9 fields:
    xaction#, date, descr, memo, amt debit, amt credit, bal, check#, fees
    
    Write an outfile that looks like...
        !Type:Bank
        C*
        Dmm/dd/yyyy
        T (negative for withdrawal)
        N###      (## = checkNumber)
        P  (description...)
        ^
    Each entry begins with "C" and ends with "^".'''

    outfile.write('!Type:Bank\n')

    reader = csv.reader(infile)
        
    for arow in reader:
        if len(arow) == 1:
            continue                    # a couple of title lines
        if len(arow) != 9:
            print("I don't grok this row with len=%d:\n\t'%s'" %
                        (len(arow), "',".join(arow)), outfile=sys.stderr)
            continue
        trans_num, date, descr, memo, deb, cred, bal, cnum, fees = arow
        if date.startswith('Date'):
            continue                    # that's the header line
        outfile.write('C*\n')
        outfile.write('D%s\n' % date)
        if deb:
            outfile.write('T%s\n' % deb)
        else:
            outfile.write('T%s\n' % cred)
        if cnum:
            outfile.write('N%s\n' % cnum)
        outfile.write('P%s\n' % descr)
        outfile.write('^\n')
    sys.exit(0)

if __name__ == '__main__':
    try:
        infile = open(sys.argv[1], 'r')
        outfile = open(sys.argv[2], 'w')
    except:
        print('usage: %s INFILE OUTFILE' % os.path.basename(sys.argv[0]))
    main(infile, outfile)
UPDATE 2014-06-24: Added a "C*" at the beginning of each entry. Without this, my quicken "check register" wasn't showing the "checks" as being Cleared.