Generating VGA video with Verilog

I’ve started work on an FPGA based Z80 computer. I wanted to generate the video in the FPGA as well, turns out VGA is fairly easy to generate. There’s numerous websites that describe the timing and parameters of the various VGA modes, and how VGA works.

This project is also my Verilog learning experience, so pardon any horrible code.

In a nutshell, for basic VGA you need a total of 5 digital output lines. One for horizontal sync, one for vertical, and one each for the red, green, and blue video signals. The RGB signals are actually analog, but for my purposes implemented as digital, either on or off. That allows a total of eight colors, including white and black. For now the text is only color. To produce additional colors, the RGB lines can be driven with a D/A converter, which could be as simple as a resistor ladder circuit.

These analog voltages need to be limited to under 1 volt. The FPGA I/O outputs are 3.3 volts so a simple resistor and diode circuit was used for each line. Here’s the circuit, as you can see it is quite simple. Virtually any reasonably fast diode can be used:

I decided to use the standard 640 x 480 60 Hz mode, as it would be more than adequate for my goal of displaying 25 lines of 80 characters.

Each character is displayed as a 8 by 16 pixel matrix, resulting in 640 x 400 pixels, the rest of the screen is not used.

The design was implemented on a RioRand EP2C5T144 Altera Cyclone II FPGA Mini Development Board available at Amazon for $19.99. What a deal.

I used the free version of Altera’s Quartus II software package.

Video memory is a 2K byte dual ported RAM, organized in a linear fashion, using 2000 bytes.

The character generator ROM is a 1K ROM, storing 8 rows of 8 pixels for each of the 128 ASCII characters. Each row is displayed twice, for a total of 16 rows. As of now the high bit of the stored ASCII value in video memory is not used, I plan on eventually using it to implement a low-res color graphics mode, a la the Apple ][.

The RioRand-EP2C5T144 FPGA dev board has a 50 MHz oscillator signal as a clock input. I implemented a 4x PLL to produce an internal 200 MHz clock. While the 50 MHz clock is adequate for 640×480 video, the faster clock will allow higher resolution video modes down the road.

A state machine is used to generate the video. A frame of video actually has 525 scan lines, it starts with 10 “front porch” lines (essentially all black video lines), then 2 lines of vertical sync, followed by 33 more “back porch” lines.
Front porch and back porch are old terms from the original NTSC analog TV system developed in the 1940s. You’ll see that totals 45 lines, leaving 480 lines of actual video.

Each video line has 800 pixels (as far as timing is concerned). There’s a front porch of 16 pixels, 96 pixels of horizontal sync, 48 pixels of a back porch, then 640 pixels of actual video content. At a 200 MHz clock, each pixel is 8 clock cycles.

Prior to the display of a pixel, the video RAM is read. The lower 7 bits of the byte, which is the ASCII value of the character to be displayed, is used along with 3 bits of the video line, to form a 10 bit address into the character generator.

Since each line of a character is displayed twice, bits [3:1[ and not [2:0] of the video line are used. If I was willing to dedicate 2K to the character ROM, I could display 16 rows of pixels for each character.

The resulting VGA video, the video RAM was pre-loaded with data using the Altera MegaWizard option to do so from an Intel Hex file.

Verilog Code:


module vga1
	(
	clock_in,
	
	red,
	green,
	blue,
	hsync,
	vsync
	);


// these are placeholders for when we hook up a CPU later
reg [15:0] addressWriteVideoRam=16'b0;
reg [7:0] dataInWriteVideoRam;
wire [7:0] dataOutReadVideoRam;
reg writeEnableWriteVideoRam=1'b0;
reg writeVideoRamClock;


// VGA video generator, undocumented for now. Good luck. 
	
input clock_in; // 50 MHz input clock

// RGB outputs
output reg red;
output reg green;
output reg blue;

// horizontal and vertical sync outputs
output reg hsync;
output reg vsync;

wire pllClock;
pll my_pll (clock_in, pllClock); // 200 MHz clock from the 50 MHz board clock


reg	[9:0]  char_rom_address;
wire	[7:0]  char_rom_output;
reg char_rom_clock;
reg [7:0] char_rom_data_byte;

reg pixel;
reg [15:0] clockCounter; // 200 MHz clock
reg [15:0] pixelCounter; // 25 MHz pixel clock
	
reg video_inclock;	// video RAM clock
reg video_wren; // not used yet
reg [7:0] video_data; // not used yet

// 1K x 8 bit video character ROM, 8 lines of 8 pixels for 128 characters, high ASCII bit not used
char_rom my_char_rom(char_rom_address, char_rom_clock, char_rom_output);

// 2K video dual port RAM for 25 lines of 80 characters	
video_ram_dp the_video_ram(.address_a (video_display_address[10:0]), .data_a(video_data), .clock_a(video_inclock), .wren_a(video_wren), .q_a(video_data_byte) ,
	.address_b(addressWriteVideoRam[10:0]), .data_b(dataInWriteVideoRam), .clock_b(writeVideoRamClock), .wren_b(writeEnableWriteVideoRam), .q_b(dataOutReadVideoRam)  );

	
reg [15:0] video_display_address;
wire [7:0] video_data_byte;

reg [15:0] horzCounter=16'd0;
reg [15:0] lineCounter=16'd0;


// VGA640x480x60_200mhz clock
parameter pixel_rate=8;
parameter horz_front_porch=16*pixel_rate;  
parameter horz_sync=96*pixel_rate;  		
parameter horz_back_porch=48*pixel_rate;
parameter horz_line=800*pixel_rate;
parameter vert_front_porch=10;
parameter vert_sync=2;
parameter vert_back_porch=33;
parameter vert_frame=525;
parameter horz_sync_polarity = 1'b0;
parameter vert_sync_polarity = 1'b0;
parameter first_video_line=49; // first line of video
parameter number_of_video_lines=400; // 25 ASCII lines of 16 pixels each is 400 video lines



always @(negedge pllClock)
	begin
	char_rom_clock<=horzCounter[0];
			
end
	
always @(posedge pllClock)
	begin

	clockCounter<=clockCounter+1'b1;
	if (clockCounter[2:0]==7) pixelCounter<=pixelCounter+1'b1;

	if (pixelCounter[2:0]==7) begin
		case (clockCounter[2:0])
//		0 : not used
//		1 : not used
		3 : char_rom_address<={video_data_byte[6:0],lineCounter[3:1]}; // high 7 bits of charactor ROM address is the ASCII character, low 3 bits video line counter, we display each line twice
		4 : char_rom_data_byte<=char_rom_output; // latch the character ROM data
//		5 : not used
		6 : video_inclock<=0;	// clock video for next ASCII character
		7 : video_inclock<=1;
		endcase
	end
	
	if (horzCounter==0)	begin  // set video to black
		red<=1'b0;
		green<=1'b0;
		blue<=1'b0;
		clockCounter<=0;
		
		end
		
	if (horzCounter==horz_front_porch)	hsync<=horz_sync_polarity;
	if (horzCounter==(horz_front_porch+horz_sync))	hsync<=!horz_sync_polarity;

	if ( (lineCounter>=(vert_front_porch+vert_sync+vert_back_porch+first_video_line)) && (lineCounter<=(vert_front_porch+vert_sync+vert_back_porch+first_video_line+number_of_video_lines)) )  begin // video frame time

		if ((pixelCounter[2:0]==6) && (clockCounter[2:0]==7)) video_display_address<=video_display_address+1'b1;


		if (horzCounter==(horz_front_porch+horz_sync+horz_back_porch))	begin
			pixelCounter<=0; // start of video line
			end
			
		if (horzCounter>=(horz_front_porch+horz_sync+horz_back_porch))	begin // video line
			
			case (pixelCounter[2:0])
			0 : pixel=char_rom_data_byte[0];
			1 : pixel=char_rom_data_byte[1];
			2 : pixel=char_rom_data_byte[2];
			3 : pixel=char_rom_data_byte[3];
			4 : pixel=char_rom_data_byte[4];
			5 : pixel=char_rom_data_byte[5];
			6 : pixel=char_rom_data_byte[6];
			7 : pixel=char_rom_data_byte[7];
			endcase
			
			// set RGB outputs
			red<=pixel;
			green<=pixel;
			blue<=pixel;
			
			end  // video line time
			
	end // video frame time

	horzCounter <= horzCounter + 16'd1;
	if (horzCounter==horz_line) begin // end of scan line, set video to black
		red<=1'b0;
		green<=1'b0;
		blue<=1'b0;

		horzCounter<=0;
		lineCounter<=lineCounter+1'b1;
		
		if (lineCounter>=(vert_front_porch+vert_sync+vert_back_porch)) begin // video frame time
			if (lineCounter[3:0]!=4'd15) video_display_address<=video_display_address-8'd100; else video_display_address<=video_display_address-8'd20;  // 80 chars back to beginning of line
		end
		
		if (lineCounter==vert_front_porch) vsync<=vert_sync_polarity;
		if (lineCounter==(vert_front_porch+vert_sync)) vsync<=!vert_sync_polarity;
		
		if (lineCounter==vert_frame) begin	// end of the video frame, start over
			video_display_address<=466;		// offset so we start reading video data at the correct address
			lineCounter<=0;
			
			end
			
		end
	
	
	end

	
	
	
endmodule

Z80 Emulation

My goal is to built a Z80 system along the lines of some of the early CP/M machines. A long long time ago in a far away former life, I worked in industrial controls, and some of the systems were Z80 based. So let’s see how much I still remember.

To get things started, I decided to first go down the emulator route before building any real hardware. I grabbed a copy of z80emu and got it running in xCode in an hour or so. It came with some text ROM files that exercise the emulation, to make sure everything is working correctly.

I found this site: Z80 Monitor Type Operating System and SBC where the author build a very simple Z80 system (Z80, RAM, ROM, 8251 UART).
So for the next step, I got that ROM working in the emulator, and modified the code to route UART output to the stdout. So far so good. Then I kludged something together to read input from XCode’s debug terminal, which also worked. Yay.

Then changing gears a but I wanted to experiment accessing a SRAM chip. I connected a 32Kx8 SRAM to an Arduino Mega 2560 and verified I could write and then read back correctly from several memory locations. This was not emulating a Z80 or
anything, just testing I/O from the Arduino.

Northern Carroll County Snowfall

A comparison of the seasonal snowfall between here at my house (extreme north central Carroll County) and the COOP station in Millers (extreme northeast Carroll County).

We’re both at about the same elevation, Millers is 860 feet. I believe it is around 800 ft here at the house, although we have lots of nearby hills and valleys.

Comparing the seasons of common data, we seem to track each other very closely, so I think it may be a good proxy for my years of missing data, and gauging what the snowfall range is here from season to season. Miller’s has a mean snowfall of 34.1″.

Want to guess how much snow I’ll get this year? Enter the contest!

Season CDS Millers
2018-2019 37.8″” 35.3″
2017-2018 34.1″ 33.4″
2016-2017 22.9″ 19.6″
2015-2016 43.7″ 43.0″
2014-2015 44.7″ 47.7″
2013-2014 78.0″ 83.4″
2012-2013 30.2″
2011-2012 18.1″
2010-2011 35.5″
2009-2010 (Note 1) 87.6″
2008-2009 18.6″
2007-2008 18.0″
2006-2007 22.3″
2005-2006 28.0″
2004-2005 30.9″
2003-2004 37.5″
2002-2003 (Note 2) 67.4″
2001-2002 8.2″
2000-2001 29.4″
1999-2000 30.9″
1998-1999 25.7″
1997-1998 18.0″
1996-1997 22.2″
1995-1996 (Note 3) 80.5″
1996-1997 22.2″
1994-1995 12.0″
1993-1994 44.1″
1992-1993 50.2″
1991-1992 10.4″
1990-1991 23.3″
1989-1990 33.8″
1988-1989 13.3″

Note 1:
The infamous Winter Of 2009-2010. We had the three large blizzards, one in December (I recall 30″) and two in February (I recall 24-30″ each). Millers reports 29.9″ in December and 52.2″ in February.
After the February storms:


Note 2:
Millers reports 35.9″ in February 2003. We had a large snowfall here. Pictures:

Note 3:
My house was built Nov 1995 – May 1996. I remember visiting the construction site many times over the winter, and there was often snow.

Winter 2019-2020 Snowfall

High atop Parr’s Ridge in northern Carroll County, MD at about 800 ft elevation.

Contest!
Guess The Amount of Snow At Casa De Smolinski (and DCA/BWI/IAD)

Seasonal Snowfall Contest Entries

26.6	solarflair
29.7	cameraman  
30.5	Groveton  
34.3	Autumn Forge  
34.8	83worldtraveler  
35.7	Snownomore  
37.9	WGsnowchic   
42.4	Terpie Cat  
44.8	beadalou  
45.7	a. vagueanswer   
47.7	Kate commenting  
50.1	walter-in-fallschurch   
52.3	weatherjunkie   
54.5	eric654   
71.9	chrisofthebeagles  
77.7	merry mildest  
77.7	surewhynot  
120.1	The Adorable Miniature Snowplows  

October Total: 0.0″

Friday November 8, 2019
Morning snow flurries.

Sunday November 24, 2019
Morning snow flurries.

November Total: 0.0″

Sunday December 1, 2019
Sleet mixed with rain.

Monday December 2, 2019
Morning snow flurries. Some accumulation / dusting on the grass.

Wednesday December 4, 2019
Morning snow, 1.0″ accumulation



Wednesday December 11, 2019
Morning snow, 1.3″ accumulation




Monday December 16, 2019
Morning snow, 1.2″ accumulation

Wednesday December 18, 2019
Afternoon snow flurries, no accumulation.
“Snow Squall Warning” from the NWS around 8 pm, received 0.3″ of snow.

December Total: 3.8″

Tuesday January 7, 2020
Afternoon snow, 5.2″ accumulation

Wednesday January 8, 2020
Morning snow squalls (NWS warned), 1.2″ accumulation

Time lapse video of snowfall:

Video of squall:

Video 2:

Thursday January 16, 2020
Afternoon snow flurries.

Saturday January 18, 2020
0.2″ snow, then sleet and freezing rain.

Sunday January 19, 2020
Morning and afternoon snow flurries.

Friday January 31, 2020
Afternoon and evening snow mixing with rain, no accumulation.

January Total: 6.6″

Friday February 28, 2020
Snow flurries.

Saturday February 29, 2020
Snow flurries.

February Total: 0.0″

Friday April 10, 2020
Snow flurries.

2019-2020 Season To Date Total: 10.4″

Previous seasons:
Winter 2018-2019 37.8″
Winter 2017-2018 34.1″
Winter 2016-2017 22.9″
Winter 2015-2016 43.7″
Winter 2014-2015 44.7″
Winter 2013-2014 78.0″

Winter 2009-2010: No details, but the seasonal total was about 100″ with three major blizzards.

Setting up OP25 Scanner Trunking on an Ubuntu Virtual Machine

After a multi day (week?) saga of trying to get op25 to run on a Raspberry Pi, I decided to give it a try on a linux virtual machine, and had much better results. For the radio hardware I used one of the ubiquitous RTL SDR Dongles.

https://www.amazon.com/gp/product/B0129EBDS2/ref=ox_sc_act_title_1?tag=blackcatsyste-20

I used this guide as a reference / starting point: https://www.hagensieker.com/wordpress/2018/07/17/op25-for-dummies/

Here’s what I did to get things running:

I’m using VirtualBox for the VM setup. I did it on macOS, it should work the same way under Windows.

Create a new linux VM. I gave it 8G of RAM (perhaps overkill) and a 30G volume.

Download the Ububtu installation ISO: https://www.ubuntu.com/download/desktop

Go to Settings -> Storage for the new VM, select the ISO as the optical disc image.

Boot the VM, and install. I won’t go into the installation details, in general I found the defaults worked fine.

I installed the Guest Additions so I could cut and paste between the OS and VM.

Next I installed gqrx so I could check out that the RTL Dongle was working. The stock Ububtu I installed did not come with it:

sudo apt install gqrx-sdr

I plugged in the RTL Dongle, went to Devices -> USB in the VM menu, and assigned the dongle to the VM for use. (Don’t forget this step!)

I ran gqrx, and verified the dongle was working.
gqrx

Next, before installing op25, I had to install git:

sudo apt install git

Then grab op25:

git clone https://github.com/boatbod/op25.git

Switch to the op25 directory that was just created:

cd op25

And install it:

./install.sh

Then install gnuplot so you can see the spectrum and constellation plots:

sudo apt-get install gnuplot-x11

Go to the apps directory for op25:

cd op25/gr-op25_repeater/apps

Next you need to go to https://www.radioreference.com/ and locate the details on the trunking system for your area. Several control frequencies may be listed for your system, you need to find the currently active one. In my case, I checked them all until I found one frequency that was continuously transmitting, 852.9375 MHz.

Now, the next part was perhaps the most difficult, you need to determine the correct ppm error for your RTL dongle. All dongles seem to be off, some more than others. Mine turned out to be off by a LOT. The other tutorials I read gave examples with ppm errors of around 2 or 3. I spent a lot of time trying small values like that, and even up to 20 or 30, without success.

I brought up the spectrum plot in op25 (hit the 1 key) and looked at the spikes, representing transmissions, and checked them against my R-7000 receiver. It was confusing at first, trying to match things up. I eventually realized my dongle was off by a huge factor – about 150 kHz at 853 MHz. I ended up using a ppm value of 173, and that seems to be working. Your value will likely be different, but carefully use the spectrum plot to determine what it is, or at least get close. Then you can iterate up and down by 1 ppm. Another recommendation I read, and used, was to set the offset (used to avoid the 0 Hz spike) to zero for initial testing.

Here’s the command to run op25 with a control frequency of 852.9375 MHz, ppm of 172, and an offset of 0 Hz:

./rx.py –args ‘rtl’ -N ‘LNA:47’ -S 2400000 -f 852.9375e6 -o 0 -q 172

I found I still need to use the , and . keys to shift the received frequency offset around until the program started to decode data correctly (the tsbks value will start incrementing). Again I used the spectrum plot to
help center the control frequency.

When properly tuned, the constellation plot looks like this, hit the 2 key to bring it up:

Once that worked, the next step was to find the NAC value, which is displayed in the op25 program, in my case it was 0x661.

In the apps directory, open the trunk.tsv file in the LibreOffice editor built into Ubuntu, it opens as a spreadsheet. I edited it as follows, entering in a system name, setting the control channel and NAC values. I left the modulation alone (CQPSK) and entered a new tags file name, we’ll create that file next.

I then duplicated the tompkins.tsv file, renamed the duplicate carroll.csv to match what I entered in trunk.tsv, and then opened it in LibreOffice.

It’s a bit tedious, but you have to enter in each talkgroup tag number and name. I just went down the list of talkgroups in radio reference, and it took a few minutes. Part of the list:

Once that was done, I ran op25 again. You can append 2> followed by a filename, to route error messages to a file, so they do not clutter the screen:

./rx.py --args 'rtl' -N 'LNA:47' -S 2400000 -o 25000 -q 181 -T trunk.tsv -V -2 -U 2> stderr.2

I am using an offset of 25 kHz (25000 Hz), and notice I now had to change the ppm to -181, the RTL dongle drifted that much in a few hours!

Update, I also got it working with the AirSpy, which turned out to be very easy. I just had to install the AirSpy support with:

sudo apt install airspy

Running it is as easy as:

./rx.py --args "airspy" -q 3 -N 'IF:12,MIX:12,LNA:12' -S 2500000 -V -2 -U -T trunk.tsv

As you can see, the AirSpy is much more accurate, the ppm value is only 3.

I still need to optimize the gain settings, but this is working nicely. Much better reception than the RTL dongle, as you can imagine. Hmm… unfortunately, op25 is freezing after a while with the AirSpy. Need to investigate this…

Another update:

I decided to install Ububtu on an older i3 laptop. I resized the Windows 10 drive, and freed up 200GB (perhaps excessive, but the drive is 750GB, and I don’t really use the laptop much anymore for Windows) for the linux partition.

I followed the above steps and got gqrx running first, then op25. I have not tried with the AirSpy yet, but even with the RTL dongle, things are improved, the audio quality and overall reception are noticeably better.

Winter 2018-2019 Snowfall

High atop Parr’s Ridge in northern Carroll County, MD at about 800 ft elevation.

Contest!
Guess The Amount of Snow At Casa De Smolinski (and DCA/BWI/IAD)

Seasonal Snowfall Contest Entries

TSMagnum  26.9
Groveton  28.8
Snowbrow  35.2
mjbuffettfan  38
BigCountry  42.2
beadalou  42.3
Terpiecat  42.7
Xtrain21  44.4
ck5416  45.1
taylort2  45.8
(no name) 47.2
cdklktr  47.5
alrob8  48.3
Sneakyfeets  48.5
SSB  51.3
Sarandipity885  53.4
eric654  54.5
gelezinis vilkas  56.7
83worldtraveler  58.7
Kate commenting  59.3
asimovian  61.3
nomini  63.2
parksndc  64.7
I Love Terpiecat  65.0
cameraman  65.4
Snownomore  69.9
Autumn_Forge  67.3
chrisofthebeagles  71.9
Rex Block  73.3
Merry Mildest  75.0
walter-in-fallschurch  75.1
WBsnowchic  77
surewhynot  79
speedo311yo  88.0
PBH  88
Scrabble Girl   98.6
Chris Smolinski 100.0
The Adorable Miniature Snowplows  101

Mean: 61.0
Median: 59.3

(No, I can’t win my own contest, I thought it would to fun to add my own guess, which is just the approximate total snowfall the last time we had a solar minimum winter)

October Total: 0.0″

Thursday November 15, 2018 – Friday November 16, 2018:
7.0″ of heavy snow on the 15th.
An additional 1.25″ of snow measured early on the morning of the 16th, which fell overnight

Wednesday November 28, 2018
Snow flurries. No accumulation.

November Total: 8.25″

December Total: 0.0″

Wednesday January 9, 2019 – Thursday January 10, 2019
A dusting of snow.

Saturday January 12, 2019 – Sunday January 13, 2019
2.5″ of snow

Thursday January 17, 2019 – Friday January 18, 2019
1.5″ of snow

Saturday January 19, 2019
0.5″ of snow,then sleet, then rain.

Friday January 25, 2019
Morning snow flurries.

Tuesday January 29, 2019
3.5″ of fairly heavy snow.

Wednesday January 30, 2019
0.4″ from a snow squall.

January Total: 8.4″

Friday February 1, 2019
2.0″ of dry snow.

Monday February 11, 2019 – Tuesday February 12, 2019
5.5″ snow, ending as sleet and rain.

Sunday February 17, 2019 – Monday February 18, 2019
Glaze of freezing rain.

Wednesday February 20, 2019
4.5″ of wet snow, followed by some freezing mist. About 0.1″ ice build up on trees.

February Total: 12.0″

Friday March 1, 2019
2.9″ of snow.

Saturday March 2, 2019
1.5″ of wet snow, also some rain and sleet.

Sunday March 3, 2019
4.75″ of heavy snow.

March Total: 9.15″

2018-2019 Season To Date Total: 37.8″

Previous seasons:
Winter 2017-2018 34.1″
Winter 2016-2017 22.9″
Winter 2015-2016 43.7″
Winter 2014-2015 44.7″
Winter 2013-2014 78.0″

Winter 2009-2010: No details, but the seasonal total was about 100″ with three major blizzards.

Variation In Precipitation At Dulles International Airport (IAD) 1964-2017

Previously I wrote about the variation In precipitation At Washington Reagan National Airport (DCA) and Baltimore Washington International Airport (BWI) to see if there was a significant change over the years due to Climate Change, and now it’s time to look at Dulles International Airport.

The IAD dataset starts later than that for DCA and BWI, and starts from April 1, 1960. As with the DCA and BWI data, there appear to be a few days with missing data, as for some years there is only data for 364 days. The actual number of days per year of data was taken into account when computing means. Also, large amounts of data are not available for various dates 1960 through 1963, and of course there is no data for the remainder of 2018. So the plots are from 1964 through 2017.

Each of the graphs can be clicked to be viewed full size.

The first graph is the average daily precipitation. This is total amount of rain per year divided by the number of days in the year. This graph shows a slight increase, from 0.110 to 0.115 inches per day:

Next is a graph of the number of days per year with precipitation, followed by its inverse, the number of days without. The number of days with precipitation has increased from about 114 to 121, or 6 more days. The increases at DCA and BWI were 5 days, and they was over a larger number years. It currently rains about 115 days per year at DCA, and 118 days per year at BWI.

This leads to the next graph, the average rainfall on days when it actually rained. There is no apparent change at all – this graph is flat. So while it rains about 6 more days a year at IAD, the amount of rain, on days when it does rain, is the same. Actually if you squint, it looks like it might be very so slightly decreasing, but it is subtle. And probably going to be different once you add in another year’s worth of data.

Next, we can look at the standard deviation in rainfall amounts on days when it rains, standard deviation is amount of variation of a set of data values around the mean (average). My previous post has a link for more information about standard deviation, as well as the best standard deviation joke for math nerds.

The standard deviation rose from about 0.48 to 0.50, about half the increase seen at BWI. By comparison, the standard deviation was unchanged at DCA. If you torture the data enough, you can find possible reasons for the change. I notice there was a step change / increase in the standard deviation from 2005-2014, which seems to have since stopped. You can also spot a period in the 1990s when the standard deviation was low. I am sure 20 people can find 20 more possible reasons. It’s like a Rorschach test for meteorologists. Here’s the standard deviation plot:

Finally, we can look at the number of days that rained and had rainfall over two standard deviations (using the standard deviation for that year), treating this as significant or extreme rainfall events. The trend of this is completely flat:

To summarize the analysis of rainfall events at DCA, BWI, and IAD:

  • The number of days with rain is increasing, 5 or 6 more days per year, over the last half century or more, at all three airports.
  • The average annual measured rainfall is either decreasing (DCA) or increasing (BWI, IAD), depending on which station you look at.
  • The average amount of rain on days when it does rain is either decreasing (DCA), increasing (BWI) or exactly the same (IAD).
  • The number of days with significantly high rainfall, exceeding two standard deviations, is either very slightly increasing (BWI) or unchanged (DCA, IAD), depending on which airport you look at.
  • Variation In Precipitation At Baltimore Washington International Airport (BWI) 1937-2018

    Previously I wrote about the Variation In Precipitation At Washington Reagan National Airport (DCA) 1945-2018 to see if there was a significant change over the years due to Climate Change, and now it’s time to look at BWI Airport.

    The BWI dataset starts earlier than tat for DCA, and runs from July 1, 1937 to July 18, 2018. As with the DCA data, there appear to be a few days with missing data, as for some years there is only data for 364 days. The actual number of days per year of data was taken into account when computing means. Also, some data is not plotted for 1945 or 2018, namely the number of days with/without rain for those years, as a full year of data is not available.

    Each of the graphs can be clicked to be viewed full size.

    The first graph is the average daily precipitation. This is total amount of rain per year divided by the number of days. It shows an increase from roughly 0.108 to 0.121 inches per day from 1939 to 2018. That’s an increase of about 12%.

    Next is a graph of the number of days per year with precipitation, followed by its inverse, the number of days without. The number of days with precipitation has increased from about 113 to 118, or 5 more days. This is the same increase we saw at DCA, although it seems rain about 3 fewer days per year at DCA vs BWI (You picked the right airport to leave near, Walter):

    That leads to the next graph, the average rainfall on days when it actually rained. This shows an increase from about 0.35 inches to 0.37 inches. (The trend at DCA was actually negative, from 0.37 to 0.35 inches in fact. Exactly opposite. Hmm…):

    Next, we can look at the standard deviation in rainfall amounts on days when it rains, standard deviation is amount of variation of a set of data values around the mean (average). My previous post has a link for more information about standard deviation, as well as the best standard deviation joke for math nerds. The standard deviation in rainfall amounts at BWI is increasing. Looking at the graph, the increase seems to be mostly due to an increase during the 2005-2015 period, which has since ended. Is it a short term variation due to random processes, or part of a long term shift?

    Next we can look at the number of days that rained and had rainfall over two standard deviations (using the standard deviation for that year), treating this as significant or extreme rainfall events. The trend of this is completely flat:

    But, you might ask, isn’t the standard deviation slightly increasing each year? What if we use a fixed standard deviation value, like 0.50 which seems to be the mean value? Doing that, you do see an increase from about 9 to 11 days per year:

    Next up… Dulles Airport (which unfortunately has the shortest dataset of all three major airports)

    Variation In Precipitation At Washington Reagan National Airport (DCA) 1945-2018

    Recently the weather, at least precipitation wise, in the DCA area has been variable. It was very wet and rainy, then we had dry conditions for several weeks with essentially no rain. Now, it is very wet again. Are we seeing extreme (some may say historic) changes in the weather? Or are these just the usual variations?

    Are rainfall events are becoming less common, but more extreme? That is, it rains less often, but we get more rain when it does rain, rather than getting rainfall spread out evenly over time as it used to be. And maybe we’re getting more rain overall. Or possibly less rain, those two claims seems to depend on recent weather memory.

    To check this claim, data for Washington Reagan National Airport (DCA) was downloaded from the NOAA NCDC site and analyzed in several ways. You can download this data yourself, if you wish: https://www.ncdc.noaa.gov/cdo-web/

    The dataset runs from July 1, 1945 to July 18, 2018. There appear to be a few days with missing data, as for some years there is only data for 364 days. The actual number of days per year of data was taken into account when computing means. Also, some data is not plotted for 1945 or 2018, namely the number of days with/without rain for those years, as a full year of data is not available.

    Each of the graphs can be clicked to be viewed full size.

    First, is there a significant long term trend in the amount of precipitation at DCA? No, there does not seem to be (if you squint you may see a very small decline over time, the sign of this slope likely changes from year to year with normal variability in rainfall):
    Mean daily precipitation

    Second, are rain events becoming less common, but with higher rainfall totals from those events? That would mean we are seeing fewer days with rain, but more rain on those days. The following two graphs show the number of days without any rain, and the number of days with rain, defined as 0.01″ or more. First the number of days without rain. Which is not increasing, but actually decreasing:
    Days per year without precipitation

    And the number of days per year with rain, which of course is just the inverse of the previous graph. It’s raining 5 or 6 more days per year (sorry, Walter):
    Days per year with precipitation

    Third, what about the rainfall totals on days when it actually rains. Is that increasing, leading to more extreme rain events? No, it isn’t. It is actually decreasing, which makes sense considering the mean rainfall per year is essentially steady, and it is raining a few more days out of the year:
    Mean precipitation for days with precipitation

    OK, maybe it is mostly the same, but we’re getting a few more extreme rainfall events per year? Let’s look at the standard deviation of the rainfall amounts, again only for days when it actually rains.

    Standard deviation is amount of variation of a set of data values around the mean (average), there is an explanation here: https://en.wikipedia.org/wiki/Standard_deviation If you get this joke, you understand standard deviation: “Yo mama is so mean she has no standard deviation”

    Hmm, no, that is also steady:
    Standard deviation of daily precipitation

    One more thing, look at the number of days with very extreme rainfall. How about the number of days where the rainfall exceeded two standard deviations? That also seems to be flat:
    Days above two standard deviations

    Anything else we can check to see if precipitation is indeed getting more extreme in Washington DC? So far, it doesn’t seem to be.

    Note, the purpose of this analysis was not to try and discredit man made climate change aka AGW, which is certainly real. Only to see if claims of a noticeable effect on the precipitation patterns in the DC area can be confirmed, which does not seem to be the case.

    Privacy Policy

    A. Introduction

    The privacy of our website visitors is very important to us, and we are committed to safeguarding it. This policy explains what we will do with your personal information.
    Consenting to our use of cookies in accordance with the terms of this policy when you first visit our website permits us to use cookies every time you visit our website.
    B. Credit
    This document was created using a template from SEQ Legal (seqlegal.com)
    and modified by Website Planet (www.websiteplanet.com)

    C. Collecting personal information

    The following types of personal information may be collected, stored, and used:

    information about your computer including your IP address, geographical location, browser type and version, and operating system;
    information about your visits to and use of this website including the referral source, length of visit, page views, and website navigation paths;
    information, such as your email address, that you enter when you register with our website;
    information that you enter when you create a profile on our website—for example, your name, profile pictures, gender, birthday, relationship status, interests and hobbies, educational details, and employment details;
    information, such as your name and email address, that you enter in order to set up subscriptions to our emails and/or newsletters;
    information that you enter while using the services on our website;
    information that is generated while using our website, including when, how often, and under what circumstances you use it;
    information relating to anything you purchase, services you use, or transactions you make through our website, which includes your name, address, telephone number, email address, and credit card details;
    information that you post to our website with the intention of publishing it on the internet, which includes your username, profile pictures, and the content of your posts;
    information contained in any communications that you send to us by email or through our website, including its communication content and metadata;
    any other personal information that you send to us.
    Before you disclose to us the personal information of another person, you must obtain that person’s consent to both the disclosure and the processing of that personal information in accordance with this policy

    D. Using your personal information

    Personal information submitted to us through our website will be used for the purposes specified in this policy or on the relevant pages of the website. We may use your personal information for the following:

    administering our website and business;
    personalizing our website for you;
    enabling your use of the services available on our website;
    sending you goods purchased through our website;
    supplying services purchased through our website;
    sending statements, invoices, and payment reminders to you, and collecting payments from you;
    sending you non-marketing commercial communications;
    sending you email notifications that you have specifically requested;
    sending you our email newsletter, if you have requested it (you can inform us at any time if you no longer require the newsletter);
    sending you marketing communications relating to our business or the businesses of carefully-selected third parties which we think may be of interest to you, by post or, where you have specifically agreed to this, by email or similar technology (you can inform us at any time if you no longer require marketing communications);
    providing third parties with statistical information about our users (but those third parties will not be able to identify any individual user from that information);
    dealing with inquiries and complaints made by or about you relating to our website;
    keeping our website secure and prevent fraud;
    verifying compliance with the terms and conditions governing the use of our website (including monitoring private messages sent through our website private messaging service); and
    other uses.
    If you submit personal information for publication on our website, we will publish and otherwise use that information in accordance with the license you grant to us.

    Your privacy settings can be used to limit the publication of your information on our website and can be adjusted using privacy controls on the website.

    We will not, without your express consent, supply your personal information to any third party for their or any other third party’s direct marketing.

    E. Disclosing personal information

    We may disclose your personal information to any of our employees, officers, insurers, professional advisers, agents, suppliers, or subcontractors as reasonably necessary for the purposes set out in this policy.

    We may disclose your personal information to any member of our group of companies (this means our subsidiaries, our ultimate holding company and all its subsidiaries) as reasonably necessary for the purposes set out in this policy.

    We may disclose your personal information:

    to the extent that we are required to do so by law;
    in connection with any ongoing or prospective legal proceedings;
    in order to establish, exercise, or defend our legal rights (including providing information to others for the purposes of fraud prevention and reducing credit risk);
    to the purchaser (or prospective purchaser) of any business or asset that we are (or are contemplating) selling; and
    to any person who we reasonably believe may apply to a court or other competent authority for disclosure of that personal information where, in our reasonable opinion, such court or authority would be reasonably likely to order disclosure of that personal information.
    Except as provided in this policy, we will not provide your personal information to third parties.

    F. International data transfers

    Information that we collect may be stored, processed in, and transferred between any of the countries in which we operate in order to enable us to use the information in accordance with this policy.
    Information that we collect may be transferred to the following countries which do not have data protection laws equivalent to those in force in the European Economic Area: the United States of America, Russia, Japan, China, and India.
    Personal information that you publish on our website or submit for publication on our website may be available, via the internet, around the world. We cannot prevent the use or misuse of such information by others.
    You expressly agree to the transfers of personal information described in this Section F.
    G. Retaining personal information

    This Section G sets out our data retention policies and procedure, which are designed to help ensure that we comply with our legal obligations regarding the retention and deletion of personal information.
    Personal information that we process for any purpose or purposes shall not be kept for longer than is necessary for that purpose or those purposes.
    Without prejudice to article G-2, we will usually delete personal data falling within the categories set out below at the date/time set out below:
    personal data type will be deleted after 24 months.
    Notwithstanding the other provisions of this Section G, we will retain documents (including electronic documents) containing personal data:
    to the extent that we are required to do so by law;
    if we believe that the documents may be relevant to any ongoing or prospective legal proceedings; and
    in order to establish, exercise, or defend our legal rights (including providing information to others for the purposes of fraud prevention and reducing credit risk).
    H. Security of your personal information

    We will take reasonable technical and organizational precautions to prevent the loss, misuse, or alteration of your personal information.
    We will store all the personal information you provide on our secure (password- and firewall-protected) servers.
    All electronic financial transactions entered into through our website will be protected by encryption technology.
    You acknowledge that the transmission of information over the internet is inherently insecure, and we cannot guarantee the security of data sent over the internet.
    You are responsible for keeping the password you use for accessing our website confidential; we will not ask you for your password (except when you log in to our website).
    I. Amendments

    We may update this policy from time to time by publishing a new version on our website. You should check this page occasionally to ensure you understand any changes to this policy. We may notify you of changes to this policy by email or through the private messaging system on our website.

    J. Your rights

    You may instruct us to provide you with any personal information we hold about you; provision of such information will be subject to the following:

    the payment of a fee to be determined based on the cost to provide the data; and
    the supply of appropriate evidence of your identity, we will usually accept a photocopy of your passport certified by a notary plus an original copy of a utility bill showing your current address.
    We may withhold personal information that you request to the extent permitted by law.

    You may instruct us at any time not to process your personal information for marketing purposes.

    In practice, you will usually either expressly agree in advance to our use of your personal information for marketing purposes, or we will provide you with an opportunity to opt out of the use of your personal information for marketing purposes.

    K. Third party websites

    Our website includes hyperlinks to, and details of, third party websites. We have no control over, and are not responsible for, the privacy policies and practices of third parties.

    L. Updating information

    Please let us know if the personal information that we hold about you needs to be corrected or updated.

    M. Cookies

    Our website uses cookies. A cookie is a file containing an identifier (a string of letters and numbers) that is sent by a web server to a web browser and is stored by the browser. The identifier is then sent back to the server each time the browser requests a page from the server. Cookies may be either “persistent” cookies or “session” cookies: a persistent cookie will be stored by a web browser and will remain valid until its set expiry date, unless deleted by the user before the expiry date; a session cookie, on the other hand, will expire at the end of the user session, when the web browser is closed. Cookies do not typically contain any information that personally identifies a user, but personal information that we store about you may be linked to the information stored in and obtained from cookies.
    We use both session and persistent cookies on our website.

    The names of the cookies that we use on our website, and the purposes for which they are used, are set out below:
    we use Google Analytics and Adwords on our website to recognize a computer when a user visits the website, track users as they navigate the website, enable the use of a shopping cart on the website,
    improve the website’s usability, analyze the use of the website, administer the website, prevent fraud and improve the security of the website, personalize the website for each user,
    target advertisements which may be of particular interest to specific users.
    Most browsers allow you to refuse to accept cookies—for example:
    in Internet Explorer (version 10) you can block cookies using the cookie handling override settings available by clicking “Tools,” “Internet Options,” “Privacy,” and then “Advanced”;
    in Firefox (version 24) you can block all cookies by clicking “Tools,” “Options,” “Privacy,” selecting “Use custom settings for history” from the drop-down menu, and unticking “Accept cookies from sites”; and
    in Chrome (version 29), you can block all cookies by accessing the “Customize and control” menu, and clicking “Settings,” “Show advanced settings,” and “Content settings,” and then selecting “Block sites from setting any data” under the “Cookies” heading.
    Blocking all cookies will have a negative impact upon the usability of many websites. If you block cookies, you will not be able to use all the features on our website.

    You can delete cookies already stored on your computer—for example:
    in Internet Explorer (version 10), you must manually delete cookie files (you can find instructions for doing so at http://support.microsoft.com/kb/278835 );
    in Firefox (version 24), you can delete cookies by clicking “Tools,” “Options,” and “Privacy”, then selecting “Use custom settings for history”, clicking “Show Cookies,” and then clicking “Remove All Cookies”; and
    in Chrome (version 29), you can delete all cookies by accessing the “Customize and control” menu, and clicking “Settings,” “Show advanced settings,” and “Clear browsing data,” and then selecting “Delete cookies and other site and plug-in data” before clicking “Clear browsing data.”
    Deleting cookies will have a negative impact on the usability of many websites.