Friday, September 12, 2014

42% “efficient”

Suppose a team of coders work on a project, and we say we want code reviews, or maybe inspections. We have estimates for how long various parts will take to code -- adding up to 3 engineer-weeks, say. How much effort (engineer-days) should we allocate to cover code reviews, discussion and rework?

Well, that depends of course on how much time we think it will take to review and discuss and rework the code. Let's suppose for a moment that if my colleague spends a week coding and doing some basic testing, then I will stare at the code for two days. Actually, let's say there are two reviewers, so two of us will stare at the code for two days each.

Then we have a day where three of us meet (in the morning, say) and talk about the code for a while. After 1½ or 2 hours, we go back to our desks and think some more about what we heard, and the person who wrote the code makes some changes. We repeat that but at the end of the day we're all happy with the code.

As I write this, that seems like a lot of review time, but if we do it that way, the total engineer-days spent add up thus:

  • 5 engr-days coding
  • 4 engr-days reviewing (2 reviewers @ 2 days each)
  • 3 engr-days discussion and rework (3 participants @ 1 day each)
Twelve (12) days total, for a thing whose coding estimate was 5 days.

That works out to 5/12 ≈ 42%. In other words, if you have 10 engineer-days available, then at the above rates of "overhead" you can spend only 4.2 days coding; the rest is review, discussion, rework. That doesn't include integration or system test, etc.

Perhaps I have the wrong ratio of (review + discussion + rework) to (code and basic test), but I'll confess to being surprised at the result of 42% given the above.

Monday, August 11, 2014

One size fits… one.

Today's Old Testament reading, from the book of Nehemiah, shows an interesting contrast in the way different people experience God. In chapter 1 of his book, Nehemiah tells us that he was cupbearer (or wine-taster) for the king in Babylon. He heard about the condition of Jerusalem and was very upset that the walls were broken down and the gates destroyed.

In chapter 2, he asks the king to send him “to the city in Judah where my fathers are buried so that I can rebuild it.” He doesn't mention the name Jerusalem (there probably was some sensitivity about that particular city), but the part I found particularly interesting is what came next:

I also said to him, “If it pleases the king, may I have letters to the governors of Trans-Euphrates, so that they will provide me safe-conduct until I arrive in Judah? And may I have a letter to Asaph, keeper of the king's forest, so he will give me timber to make beams for the gates of the citadel by the temple and for the city wall and for the residence I will occupy?”

And because the gracious hand of my God was upon me, the king granted my requests. So I went to the governors of Trans-Euphrates and gave them the king's letters. The king had also sent army officers and cavalry with me.

Nehemiah 2:7-9
What's so interesting about that? Well, maybe not so much in itself, but what I noticed was the contrast with Ezra, which was in the readings from some days past. The part of Ezra I'm thinking of was this:
There, by the Ahava Canal, I proclaimed a fast, so that we might humble ourselves before our God and ask him for a safe journey for us and our children, with all our possessions. I was ashamed to ask the king for soldiers and horsemen to protect us from enemies on the road, because we had told the king, "The gracious hand of our God is on everyone who looks to him, but his great anger is against all who forsake him."

So we fasted and petitioned our God about this, and he answered our prayer.

Ezra 8:21-23
What I take from these two men of God is not that Ezra trusted God and Nehemiah trusted in military escorts. And it's not that Nehemiah was bold and Ezra was timid. What I get is that God deals with us in a way that's suited to us. He knows us individually, knows our personalities and inclinations and strengths and weaknesses (besides our joys and tears and aspirations and disappointments).

He does not treat us like interchangeable parts; he knows us and cares for each one of us. And the way we honor him is something that's unique to each one of us too. How do I love my children? I love each one for who she is. And that gives me a little picture, a little sense of appreciation for how God loves me—he loves me in a way that's different from the way he loves you, and our paths will be different, and that's okay; indeed, it is very good.

Thursday, June 26, 2014

Disable new user signups (registration spam) on a wordpress blog

The lovely Carol has a wordpress blog. She's been getting lots of "new user registrations," and she spends too much time deleting these new "users". These are not real people; they're more likely "spammers wanting to take control of the system through multiple user registrations" according to this site.

I did a web search on "wordpress new users" or something like this (no quotation marks) and found an article titled “How to Disable New User Registration on WordPress Using PhpMyAdmin,” which was useful, except I don't have PhpMyAdmin. And I don't have cPanel either. And that was the most promising of the links that came up. Bummer.

What's a geek to do? well, this:

$ ls
dec24.html                 wp-config-sample.php
fire-safe-box/             wp-content/
INDEX.HTML.oneclickbackup  wp-cron.php
index.php                  wp-includes/
license.txt                wp-links-opml.php
md5sums.txt                wp-load.php
readme.html                wp-login.php
wp-activate.php            wp-mail.php
wp-admin/                  wp-settings.php
wp-blog-header.php         wp-signup.php
wp-comments-post.php       wp-trackback.php
wp-config.php              xmlrpc.php
$ 
H'm, wp-signup.php you say? That looked promising. I noticed this:
 30 if ( !is_multisite() ) {
 31         wp_redirect( site_url('wp-login.php?action=register') );
 32         die();
 33 }
"wp-signup.php" [readonly]
THAT looked promising. I looked in wp-login.php and searched for "register"; I found this:
646         if ( !get_option('users_can_register') ) {
647                 wp_redirect( site_url('wp-login.php?registration=disabled') );
648                 exit();
649         }
"wp-login.php" [readonly] 921 lines
O-kay! So we want to kill off the option 'users_can_register'.

By looking around, I discovered that get_option() gets things from the cache. I searched wordpress.org for "get_option" and landed at http://wordpress.org/search/get_option, which refers to the options database table. Somehow I got the idea of looking in wp-config.php where I discovered this:

// ** MySQL settings ** //
define('DB_NAME', 'bedrockdb');    // The name of the database
define('DB_USER', 'fred_flint');     // Your MySQL username
define('DB_PASSWORD', 'yabbadabbadoo'); // ...and password
define('DB_HOST', 'hanna.bar.be.ra');    // 99% chance you won't need to change this value
(Of course the names are changed to protect… well, myself.) In another window, I typed:
$ mysql
ERROR 2002 (HY000): Can't connect to local MySQL server through socket '/tmp/mysql.sock' (111)
D'oh! I had to fill in the hostname and the username and…
$ mysql -h 'hanna.bar.be.ra' -u 'fred_flint' -p
Enter password: <here I typed “yabbadabbadoo”>
Welcome to the MySQL monitor.  Commands end with ; or \g.
Your MySQL connection id is 24209555 to server version: 5.5.37-35.1-log

Type 'help;' or '\h' for help. Type '\c' to clear the buffer.

mysql> use bedrockdb
Database changed
mysql> show tables;
+-----------------------+
| Tables_in_bedrockdb   |
+-----------------------+
| wp_commentmeta        |
| wp_comments           |
| wp_links              |
| wp_options            |
| wp_postmeta           |
| wp_posts              |
| wp_subscribe2         |
| wp_term_relationships |
| wp_term_taxonomy      |
| wp_terms              |
| wp_usermeta           |
| wp_users              |
+-----------------------+
12 rows in set (0.02 sec) 
Okay, that's good. Now which table? wp_options? Yes! And according to the above article, I want wp_options. So I said:
mysql> describe wp_options;
+--------------+---------------------+------+-----+---------+----------------+
| Field        | Type                | Null | Key | Default | Extra          |
+--------------+---------------------+------+-----+---------+----------------+
| option_id    | bigint(20) unsigned | NO   | PRI | NULL    | auto_increment |
| option_name  | varchar(64)         | NO   | UNI |         |                |
| option_value | longtext            | NO   |     | NULL    |                |
| autoload     | varchar(20)         | NO   |     | yes     |                |
+--------------+---------------------+------+-----+---------+----------------+
4 rows in set (0.01 sec)

mysql> select * from wp_options where option_name = 'users_can_register'; 
+-----------+--------------------+--------------+----------+
| option_id | option_name        | option_value | autoload |
+-----------+--------------------+--------------+----------+
|         6 | users_can_register | 1            | yes      |
+-----------+--------------------+--------------+----------+
1 row in set (0.02 sec)
It's been a while since I updated a table in MySQL. Naturally I went to a web search to find a list of SQL commands. This one told me what I needed, down in "Data Commands."

Let's see, I wanted to change the option_value from 1 to 0 where… OK, I tried this.

mysql> update wp_options set option_value = 0 where option_name = 'users_can_register';
Query OK, 1 row affected (0.01 sec)
Rows matched: 1  Changed: 1  Warnings: 0
Wow, it worked the first time! How often does that happen? Just to be sure, I typed this:
mysql> select * from wp_options where option_name = 'users_can_register';
+-----------+--------------------+--------------+----------+
| option_id | option_name        | option_value | autoload |
+-----------+--------------------+--------------+----------+
|         6 | users_can_register | 0            | yes      |
+-----------+--------------------+--------------+----------+
1 row in set (0.01 sec)

mysql> quit
Bye
Now for the acid test. I went to the sign-up page "wp-signup.php" and got: "User registration is currently not allowed."

And that's what I'll do next time.

Thursday, June 12, 2014

CSV to QIF transformation with Python3

This evening I discovered that the folks at my bank, or more probably the folks running the online banking website, have "upgraded" the user interface, and they no longer support downloading transactions to the "old" Quicken file format (i.e., "QIF"). Instead, they (mostly) support "export"ing transactions to CSV.

Fortunately, a format conversion from CSV to QIF isn't that hard. I first did a web search, which yielded

  • a website (upload your csv and we'll give you a qif file),
  • some Windows (I guess) software--free to download and try, but I guess they want money for it, and
  • maybe something else that I didn't think would work for me
so I decided to just hack something together. Of course it would be in Python, and probably python3.

I remembered that there was a module that knew how to read and write CSV files; a quick look at the library reference showed that the module is called, oddly enough, "csv"; the result is described here (produced by "pydoc3 -w csv_to_qif")

 
 
csv_to_qif
index
/mnt/home/collin/projects/csv-qif/csv_to_qif.py

transform a "csv" file on stdin to a "qif" file on stdout.

 
Modules
       
csv
os
sys

 
Functions
       
main(infile, outfile)
Read infile -- we expect valid entries to have 9 fields:
xaction#, date, descr, memo, amt debit, amt credit, bal, check#, fees
 
Write an outfile that looks like...
    !Type:Bank
    C*
    Dmm/dd/yyyy
    T<amount> (negative for withdrawal)
    N###      (## = checkNumber)
    P<descr>  (description...)
    ^
Each entry begins with "C" and ends with "^".
Here's the code. It is Copyright©2014 by me, but you are free to use, redistribute and or modify it under the terms of GPL2; see below.
#!/usr/bin/python3 -utt
# vim:et
'''transform a "csv" file on stdin to a "qif" file on stdout.'''

# This program is free software: you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
# the Free Software Foundation, version 2.
# NO WARRANTY express or implied.

import csv
import os
import sys

def main(infile, outfile):
    '''Read infile -- we expect valid entries to have 9 fields:
    xaction#, date, descr, memo, amt debit, amt credit, bal, check#, fees
    
    Write an outfile that looks like...
        !Type:Bank
        C*
        Dmm/dd/yyyy
        T (negative for withdrawal)
        N###      (## = checkNumber)
        P  (description...)
        ^
    Each entry begins with "C" and ends with "^".'''

    outfile.write('!Type:Bank\n')

    reader = csv.reader(infile)
        
    for arow in reader:
        if len(arow) == 1:
            continue                    # a couple of title lines
        if len(arow) != 9:
            print("I don't grok this row with len=%d:\n\t'%s'" %
                        (len(arow), "',".join(arow)), outfile=sys.stderr)
            continue
        trans_num, date, descr, memo, deb, cred, bal, cnum, fees = arow
        if date.startswith('Date'):
            continue                    # that's the header line
        outfile.write('C*\n')
        outfile.write('D%s\n' % date)
        if deb:
            outfile.write('T%s\n' % deb)
        else:
            outfile.write('T%s\n' % cred)
        if cnum:
            outfile.write('N%s\n' % cnum)
        outfile.write('P%s\n' % descr)
        outfile.write('^\n')
    sys.exit(0)

if __name__ == '__main__':
    try:
        infile = open(sys.argv[1], 'r')
        outfile = open(sys.argv[2], 'w')
    except:
        print('usage: %s INFILE OUTFILE' % os.path.basename(sys.argv[0]))
    main(infile, outfile)
UPDATE 2014-06-24: Added a "C*" at the beginning of each entry. Without this, my quicken "check register" wasn't showing the "checks" as being Cleared.

Tuesday, May 13, 2014

Rejoice with me!

The petite woman was about to get into her car, but she was smiling and looking at me as I walked my little dog down her street. "Good morning," I said.

"Good morning!" she replied. "I was so happy…" she began, and I wondered where this was going. "… because my dog ran away, but he came back just now. I was getting ready to go to work, and there he was!" She wiggled a bit to show me how he'd greeted her.

"That's wonderful!" I said. I knew what it felt like to lose track of a pet, or a child, and to find them again.

She went on to tell me about how the dog has run away in the past, but the tag on his collar has her phone number—this time, though, she had cut his hair and left his collar off. She spoke of her futile visit to the Humane Society, and of her other dog's despondency—he had missed his buddy. "But then there he was!" she said. She could not contain her joy.

"Congratulations!" I said. "That's a great thing." We said good-bye, and I thought of a story Jesus told nearly two thousand years ago.

4“Suppose one of you has a hundred sheep and loses one of them. Does he not leave the ninety-nine in the open country and go after the lost sheep until he finds it? 5And when he finds it, he joyfully puts it on his shoulders and goes home. Then he calls his friends and neighbors together and says, 6‘Rejoice with me; I have found my lost sheep.’ 7I tell you that in the same way there will be more rejoicing in heaven over one sinner who repents than over ninety-nine righteous persons who do not need to repent.
Luke 15:4-7
The story has two points: one is that there is rejoicing in heaven when someone repents (no, really!). Our experience of joy on earth is both an echo of our Creator's joy and a hint of future joys in heaven, even as the love I feel toward my children points to the love my heavenly father feels toward me. As Merton writes,
All nature is meant to make us think of paradise. Woods, fields, valleys, hills, the rivers and the sea, the clouds traveling across the sky, light and darkness, sun and stars, remind us that the world was created as a paradise … Heaven is even now mirrored in created things.
No Man Is an Island 6.15 (p. 115)
Heaven is indeed mirrored in created things—and not just in things we can see, but also in love and joy.

The parable has a barb: The Pharisees and teachers of the law did not rejoice when "tax collectors and sinners" were listening to Jesus and presumably repenting; hence they were no friends of heaven.
The second point, and probably the reason Jesus told the parable, is this: Friends and neighbors rejoice together. That is, if someone is happy because a lost sheep or coin (or dog) is found, friends and neighbors are the ones who rejoice with them. That is, both here and in the parable of the lost coin (Luke 15:8-10), the finder calls friends and neighbors together to celebrate.

I had a chance that morning to be a neighbor to this woman, if not a friend, by rejoicing with her. Come to think of it, I'm glad we had a chance to talk, because it reminded me of the Great Shepherd's joy, which I can share in, and of the many good gifts he gives me daily.

Tuesday, April 01, 2014

I want to be this guy

I've been thinking for a while about that prayer that begins, …asking God to fill you with the knowledge of his will through all spiritual wisdom and understanding. And we pray this in order that you may live a life worthy of the Lord and may please him in every way… (Colossians 1:9-10). Last night I opened Merton's No Man Is an Island more or less at random, where I found this:
If we have the Spirit of God in our hearts, we will live by His law of charity, inclined always to peace rather than dissension, to humility rather than arrogance, to obedience rather than rebellion, to purity and temperance, to simplicity and quietness and calm, to strength, generosity, and wisdom, to prudence and all-embracing justice, and we will love others more than ourselves, for it is the commandment of Jesus that we should love one another as he has loved us (John 15:12).
Thomas Merton, No Man Is an Island 4.9 (p. 61)
Now there is a picture of the kind of guy I want to be. It's not a lot different from the description of the Spirit's fruit from Galatians 5 (love, joy, peace, patience…) but the wording is fresh (some 1900 years newer). But as with the fruit of the Spirit, this sort of life is the fruit (the result) of the Spirit; that is, it's supernatural, as Merton also explains:

None of these things can be done without prayer, and we must turn to prayer first of all, not only to discover God's will but above all to gain the grce to carry it out with all the strength of our desire.

ibid.
This put me in mind of Hebrews 4, where the author tells us that
Therefore, since we have a great high priest who has passed through the heavens, Jesus, the Son of God, let us hold fast our confession. For we do not have a high priest who cannot sympathize with our weaknesses, but one who has been tested in all things as we are, yet without sin. Therefore, let us draw near with confidence to the throne of grace, that we may receive mercy and may find grace to help in time of need.
Hebrews 4:14-16
As the old hymn says, I need thee every hour, which is a great encouragement to me to pray all the time.

Tuesday, March 25, 2014

How death helped me focus on hope in life

Uncle Harry died on February; he was about 95. On hearing that news, his brother Gilbert, nearly 97, took a nosedive and died two weeks later.

I sat with my father at Gilbert's viewing. "Every day's a gift, isn't it?" It made me think about what I hope for my final years.

Both Harry and Gilbert had had strokes; they were wheelchair-bound and had not been in good health for some years. My aunt, their younger sister, has been losing her memory for some time. If I am fortunate enough to live to 95 or 97, what do I wish for that time?

Most likely my physical condition won't be great. Even if I don't fall 10' onto a concrete floor, as Dad has, I won't be doing 20- or 30-mile bike rides at that age. And a lot of my memory will probably be gone, too; already the kids remember lots of things I have long forgotten.

David Brinkner asks, "If you could choose just one word to describe what you want people to know about you, what would it be?" What indeed? What do you or I want people to know about us—and related to that, what kind of person do you or I want to be? Putting aside the question of what remains when our memories are gone, what kind of person do I want to be?

Other than "forty years younger", what comes to mind is that I hope to finish well. I want my life to be characerized by love, joy, peace, patience, kindness, goodness and generosity, gentleness, self-control and courage. Sound like a plan? The question is, what am I doing now about it?

One important thing is to practice some of those things.

I wrote that a year ago, and since then, my brother-in-law Danny also died. He was not much older than I am today, and I wrote about him at the time. Who am I becoming? What am I doing for others? These are much more important questions than "what's my bank balance?" or "what's my raise or bonus this year?"

Here's another good one: "What am I waiting for?" We had a great sermon the other day, where our pastor pointed out that many people in the Bible had excuses, reasons they weren't "ready" for God's call: Jeremiah was too young, Abraham was too old, Moses was a poor public speaker, and so on. God uses people who aren't ready, and who fits that description better than I?