Dan Merino's Blog

Handeling and Logging Python Exceptions

danmerino — Thu, 26 Apr 2012 18:48:26 +0000

To easily track down exceptions it is important to keep track the original context and trace. To show this I have created a little example of the proper and improper way of doing this.


import traceback

class ExceptionExample:
    def __init__(self):
        self.log = None

    def lose_context(self):
        try:
            self.exceptional_method()
        except Exception, e:
            raise e

    def keep_context(self):
        try:
            self.exceptional_method()
        except Exception, e:
            raise

    def keep_context_log_incorrectly(self):
         try:
            self.exceptional_method()
         except Exception, e:
            self.log = e

    def keep_context_log_correctly(self):
         try:
            self.exceptional_method()
         except Exception, e:
            self.log = traceback.format_exc()

    def exceptional_method(self):
        raise Exception("I throw stuff at you")

    def print_log(self):
        if self.log:
            print "\t" +  str(self.log)
        else:
            print "		nothing in the log\n\n"

ee = ExceptionExample()
print "Logging an exception incorrectly:"
ee.keep_context_log_incorrectly()
ee.print_log()
print "Logging an exception correctly:"
ee.keep_context_log_correctly()
ee.print_log()
#print "Lost context - bad - check out the line number"
#ee.lose_context()
#print "Proper context - check out the line number"
#ee.keep_context()

Continuos Integration (CI) for JavaScript – Jasmine and Teamcity

danmerino — Mon, 05 Mar 2012 19:03:47 +0000

Web applications are using more JavaScript than ever before. The following image (thanks to Gineer) shows the shift in the architecture types of web applications from simple thin clients to thicker clients driven by rich presentations and client logic.

Since most of the newer web applications include business logic embedded into the client using JavaScript, we must put assurances to make sure the quality of the application is not compromised over time.

In most cases, the JavaScript side of the applications goes untested or is not automated which is a bad idea.

I have selected the following tools to be able to provide Continuous Integration to Rich Internet Applications:

Jasmine is a testing framework for JavaScript. It is usually used as a red-green tool for developing applications with either TDD or BDD. Jasmin runs on the browser for convenience to check the TDD red-green method but this makes it a bit harder to automate.

Team City is a Continuos Integration server that is extremely popular due to its flexibility to work with a wide variety of projects, it is easy to configure and it works in most environments.

Jasmine Team City Reporter is a reporter type for Jasmine. Jasmine uses a plugin based architecture where it allows different types of reporters to consume and prepare the data. The Jasmine Team City Reporter module allows Jasmine to output the data to the web page console window of the browser that runs Jasmine.

PhantomJS is a headless web-kit browser that allows execution and rendering of web content. We will use it to simulate as if we would be running Jasmine with the Team City Reporter on a website, this way we can automate the whole process.

I built a simple PhatomJS script that consumes the Jasmine Team City Reporter example, it worked great!

JavaScript file to control PhantomJS


console.log('Loading a web page');
var page = new WebPage();
//This was tricky, this is the way to open LOCAL files
var url = "file://localhost/Users/danmerino/test/teamcity_reporter.html";
phantom.viewportSize = {width: 800, height: 600};
//This is required because PhantomJS sandboxes the website and it does not show up the console messages form that page by default
page.onConsoleMessage = function (msg) { console.log(msg); };
//Open the website
page.open(url, function (status) {
    //Page is loaded!
		if (status !== 'success') {
	            console.log('Unable to load the address!');
	        } else {
                 //Using a delay to make sure the JavaScript is executed in the browser
			   window.setTimeout(function () {
					page.render("output.png");
        			phantom.exit();
 		       }, 200);
				}
});

JavaScript tests




    
    Console Reporter Spec

Team City Configuration

Team City Test Results

“Whether you think you can…” explained by the Simpsons

danmerino — Fri, 24 Feb 2012 04:45:03 +0000

The following is one of my favorite quotes:

“Whether you think you can or whether you think you can’t, you’re right.” - Henry Ford

It is a very interesting quote, and hard to explain. The quote resonated with me when I saw the quote exemplified in episode “He Loves to Fly and the D’Ohs” from the Simpsons:

[Colby helping Homer to land a plane]

Colby: Okay Homer, I don’t know anything about planes, but I know about you. You have what made America great: no understanding of the limits of your power and a complete lack of concern for what anyone thinks of you. So you’ll land that plane. And do you know why? Because I heard some guy say you couldn’t.

Homer: What! I’ll show him. I’ll show that guy!

The Value of Time

danmerino — Tue, 21 Feb 2012 00:13:34 +0000

The most important lesson I learned from graduate school is that everything can be accomplished, as long as you have enough time to dedicate to your goals. Some people’s time is more valuable than others, the important thing is that each one of us puts a price on our own time and how we use it to archive our goals.

The way I see it is that the main goal is to be very productive in a short amount of time and get things done right. This is very counter intuitive in our culture. At one time or another, we have all felt proud because we have spent countless hours in the office working on a project. In reality, this is a trick the brain plays. Sometimes we are so busy doing things that are not important, yet, we feel proud because we use time as a measure of accomplishments. Instead, we should measure our progress based in terms of productivity and returns on investment.

Common Performance Testing Mistakes

danmerino — Wed, 20 Jul 2011 08:18:25 +0000

At my lab recently we have all been benchmarking our applications. Most of our work has to deal with throughput in distributed applications. A lot of the time I see people making common mistakes when performing tests so I decided to blog about it.

Never test from the same machine, your testing program is taking away significant resources from your application!
Is concurrency being tested (depends on your goals, most likely, YES)
If testing for real world applications, network latency is a HUGE factor in performance and affects applications in many different scenarios, some quick tips are:
1. Never test using a wireless network unless that is part of your tests!
2. Make sure you are not hitting your network cap or your packets are being changed by your internet provider
3. If testing a cloud service, do not test from the same service since there could be no network/connection delay. This can be due to several factors but it could be simply because the test tool might be under the same virtual machine as the application!
4. If you are using a reverse proxy for spoon feeding, make sure you test with and without it!
Most importantly, calculate performance based on real values instead of approximations. Most of the times approximations are NOT true as they are extrapolated to higher values.

In the web, performance is very important, if you don’t think so, ask Google: http://glinden.blogspot.com/2006/11/marissa-mayer-at-web-20.html

Marissa started with a story about a user test they did. They asked a group of Google searchers how many search results they wanted to see. Users asked for more, more than the ten results Google normally shows. More is more, they said.

So, Marissa ran an experiment where Google increased the number of search results to thirty. Traffic and revenue from Google searchers in the experimental group dropped by 20%.

Ouch. Why? Why, when users had asked for this, did they seem to hate it?

After a bit of looking, Marissa explained that they found an uncontrolled variable. The page with 10 results took .4 seconds to generate. The page with 30 results took .9 seconds.

Half a second delay caused a 20% drop in traffic. Half a second delay killed user satisfaction.

Pomodoro Productivity Tools

danmerino — Mon, 20 Jun 2011 19:14:26 +0000

I have been using GTD for a couple of years and it works very well for me. To get focused on activities I use a combination of tools. Most of the tools are not bound to my computer because I found myself checking them all time. Instead, the tools are on my mobile devices because I can take a quick glance at them whenever I want.

I recommend the following tools:

Due for iPad, iPod touch or iPhone (needs iOS 4).

Pomodoro for Windows Phone 7. Here is a link to a description. The application is free and you can easily get it on the marketplace by typing “pomodoro”.

PowerShell

danmerino — Tue, 14 Jun 2011 22:14:57 +0000

PowerShell is an amazing framework designed to streamline tasks. The main benefits of PowerShell is that it built and integrated to the .NET framework and provides core functionaly to COM and WMI.

If you have done any siginificant work with Linux then most likely you are experienced with Bash Scripts and in DOS batch files, of course there are also PowerShell scripts.

In addition, in PowerShell, there are small programs called cmdlets. The programs are then compiled into a DLL which then can be loaded as part of another script. It is a way of reusing and pipelining processes (think pipes and filters pattern) with objects instead of text input!

If you are used to Bash/MS-DOS it will make you happy to know most of the typical commands have aliases and you can use them right away!

List all files = ls/dir
Current Dir = pwd/cd
Read file = cat/type
Clear screen = clear/cls
Copy = cp/copy
Move = mv/move
Delete =rm/rmdir/del
Rename = mv/ren/rename
Write = echo
List processes = ps/tlist/tasklisk

PowerShell is great because if you have some experience with .NET and scripting you are a pro right away. To conclude, here my favorite script for parsing long log files courtesy of PowerShell.com:

foreach($line in (Get-Content $env:windirwindowsupdate.log `
-ReadCount 0 -Encoding UTF8)) `
{ if ($line -like '*successfully installed*') `
{ ($line -split ': ')[-1]}}

Introduction to Windows Phone 7 Programming

danmerino — Mon, 29 Nov 2010 08:00:12 +0000

Windows Phone 7 Programming Introduction

View more presentations from danmerino.

Testing Code Contracts .NET 4.0

danmerino — Wed, 10 Nov 2010 08:26:02 +0000

Why use Code Contracts?

By providing pre-compiled code contract interfaces other developers can adhere to signatures and also expected behavior. This is specially important due to the Liskov substitution principle.

Dino Esposito wrote a great article on the topic called Code Contracts Preview: Interfaces.

Testing

To test that all the right contracts are in place a test project can be created.

Testing preconditions is possible by catching the exceptions created by those preconditions.


        [TestMethod]
        [ExpectedException(typeof(ArgumentOutOfRangeException))]
        public void ModelNegativeBlance()
        {
            Account acc = new Account()
            {
                AccountName = "NewAccount",
                //0 or positive is expected
                Balance = -99,
                CreationDate = DateTime.Now

            };
        }

Testing postconditions is a bit tricky because the exceptions raised by postconditions are not meant to be caught. Therefore plain strings have to be used.


    public static class TestHelpers
    {
        public static string ContractExceptionName = "System.Diagnostics.Contracts.__ContractsRuntime+ContractException";

    }

    public class RepositoryTests
    {
        public RepositoryTests()
        {

        }

        [TestMethod]
        public void InsertModelWithNoParitionKeyDueToBadRepository()
        {

            var repo = SetBadRepo();
            Account acc = new Account()
            {
                AccountName = "NewAccount",
                Balance = 0,
                CreationDate = DateTime.Now

            };

            try
            {
                repo.InsertAccount(acc);
            }
            catch (Exception ex)
            {
                Assert.AreEqual(TestHelpers.ContractExceptionName, ex.GetType().FullName);
            }
        }
    }

Do not use $_SERVER['PHP_SELF']

danmerino — Mon, 08 Nov 2010 06:33:57 +0000

I have seen many times that logos and links to a home page of a site use the following:

Sadly, its is very dangerous because it might carry additional trailing data. This is specially an issue when doing SEO.

For example:
It will work fine for: /moo/daaa/boo.php
It will possibly redirect to self: /moo/daa/boo.php/mooo/daa.htm

According to some people the behavior is erratic on different configurations.

To prevent all problems I recommend using __FILE__. The following solution which works on PHP 4.0.2 and up: