Info
2005-11-25 follow up
Well, I thought that bash was not the solution to the mysql loop below, and wrote it in some perl. Here's the MySQL db connector.
#!/usr/bin/perl
use strict;
use DBI;
my $db = "albums";
my $host = "localhost";
my $user = "zxqwe";my $pw = "zxqwe";my $conninfo = "dbi:mysql:$db;$host";
my $dbh = DBI->connect( $conninfo, $user, $pw );
my $sql = "select lngSessionId from hits where strValue like '%webalizer%'";
my $sth = $dbh->prepare( $sql );
$sth->execute();
my $strValue;
my $idlist;
my $lngSessionId;
$sth->bind_columns( \$lngSessionId );
while( $sth->fetch() )
{
if( length( $idlist ) == 0 )
{
$idlist = $lngSessionId;
}
else
{
$idlist = $idlist . ", " . $lngSessionId;
}
}
$sth->finish();
$sql = "select strValue from hits where lngSessionId in ( $idlist ) and intType in ( 4,5 )";
$sth = $dbh->prepare( $sql );
$sth->execute();
$sth->bind_columns( \$strValue );
while( $sth->fetch() )
{
my @parts = split( ',', $strValue );
chomp $parts[0];
if( $parts[0] != "unknown" )
{
if( $parts[0] != "80.5.148.152"
&& $parts[0] != "83.146.42.164"
&& $parts[0] != "87.81.46.28"
&& $parts[0] != "84.12.214.154"
&& $parts[0] != "127.0.0.1"
&& $parts[0] != "127.0.0.21"
&& $parts[0] != "192.168.0.1"
&& $parts[0] != "84.12.214.157" )
{
print "$parts[0]\n" ;
}
}
}
$sth->finish();
$dbh->disconnect;
This runs in about 2 seconds, it's very much more efficient than the bash code earlier. It's now a priority that I re-learn all that I've achieved in bash in perl, with a preference to repalce bash where it is practical to do so.
2005-11-22 site abusers
I'm quite sick of those who want to spam my sites. They think I'm not aware of their repeated attempts to spoil my site statistics. I've already got something running that removes their log entries, and something else that keeps their connections open, to fill their outbound connections, in a tarpitting style.
Well, was so sick of it today that I scripted something making use of mysql tables in my previous post.
$ time for i in $( mysql -u*7! -p!"£$ --batch -e "select lngSessionId from hits where strValue like '%webalizer%';" albums ) ; do mysql -u^&* -p)(*&^ --batch -e "select strValue from hits where lngSessionId = $i and intType in ( 4,5 )" albums ; done | grep -Ev "strValue|192.168|80.5.148|127.0.0" | sort | uniq > file real 8m39.190s user 1m22.980s sys 0m33.660s
As you can see from the above, it quite literally took forever to run, if I have to go through this ordeal again I shall certainly use "in (n,n,n)" as apposed to looping the select the way I have there. So all you need now is to put the following in the /etc/init.d/rc.firewall (your location might be different) script:
for i in $( cat /home/ed/blockaddr ) ; do $IPTABLES -I INPUT -s $i -j DROP done ;
With luck I'll only have to do this once or twice more and the swine who controls the bots will give up, or he will die trying.
2005-11-18 wheres my users at!
So, you want to know where your site visitors are don't you? Well, I was asking myself the same question. I use some web log stat generators to view my site's internet presence now and then, but, perhaps it's getting out of date and perhaps it's not displaying everything that it could be. Well, I figure the most important things are the hostheader that the browser requests and the pages that the user visits within the site from their entrance. It doesn't use anything intrusive for the user, just sets a cookie integer value and checks for that cookie on each page load. Oh MySQL is required, but it could be tweaked to use plain text files if database backends are not available.
<?php
/*
CREATE TABLE `hits` (
`lngId` mediumint(8) unsigned NOT NULL auto_increment,
`datLoad` timestamp(14) NOT NULL,
`lngSessionId` mediumint(8) unsigned NOT NULL default '0',
`strValue` mediumtext NOT NULL,
`intType` int(11) NOT NULL default '0',
PRIMARY KEY (`lngId`)
) TYPE=MyISAM AUTO_INCREMENT=146 ;
*/
class pagetrack
{
var $host = "";
var $agent = "";
var $remoteport = 0;
var $remoteaddr = 0;
var $forwardaddr = 0;
var $uri;
function pagetrack()
{
$sid = cookieVar( 'sid' );
if( $sid == null )
{
$sid = $this->getmaxsid() + 1;
setcookie( "sid", $sid );
}
$host = serverVar( 'HTTP_HOST' );
$agent = serverVar( 'HTTP_USER_AGENT' );
$remoteport = serverVar( 'REMOTE_PORT' );
$remoteaddr = serverVar( 'REMOTE_ADDR' );
$forwardaddr = getenv( 'HTTP_X_FORWARDED_FOR' );
$uri = serverVar( 'REQUEST_URI' );
$referer = serverVar( 'HTTP_REFERER' );
// get the cookie sid id
if( $host != null )
{
$result = mysql_query( "insert into hits ( lngSessionId, strValue, intType ) values ( $sid, '$host', 1 )" );
}
if( $agent != null )
{
$result = mysql_query( "insert into hits ( lngSessionId, strValue, intType ) values ( $sid, '$agent', 2 )" );
}
if( $remoteport != null )
{
$result = mysql_query( "insert into hits ( lngSessionId, strValue, intType ) values ( $sid, '$remoteport', 3 )" );
}
if( $remoteaddr != null )
{
$result = mysql_query( "insert into hits ( lngSessionId, strValue, intType ) values ( $sid, '$remoteaddr', 4 )" );
}
if( $forwardaddr != null )
{
$result = mysql_query( "insert into hits ( lngSessionId, strValue, intType ) values ( $sid, '$forwardaddr', 5 )" );
}
if( $uri != null )
{
$result = mysql_query( "insert into hits ( lngSessionId, strValue, intType ) values ( $sid, '$uri', 6 )" );
}
if( $referer != null )
{
$result = mysql_query( "insert into hits ( lngSessionId, strValue, intType ) values ( $sid, '$referer', 7 )" );
}
}
function getmaxsid()
{
$retVal = "";
$result = mysql_query( "select max(lngSessionId) as a from hits" );
if( $line = mysql_fetch_array( $result, MYSQL_ASSOC ) )
{
$retVal = $line['a'];
}
return( $retVal );
}
}
$c = new pagetrack();
?>
2005-11-17 lottery stats
It's been quite some time, almost two years since I had an idea for graphing past lottery results, but I've finally got around to making a start on the applet! Stay tuned - it will be released on www.lotterystats.co.uk once I have finished all the work and it's somewhere reasonable.
The only real pain with this is that I have to put the results up somewhere that the applet can get to them and I don't fancy using up more of the available upload bandwidth here just to help you guys make money.
In recent observations and my highly scientific research, I conclude that Opera is light weight in comparison to Firefox, and has better render timings. Just everything else sucks, div tags and fonts are so different to the other major browsers.
Oh and I'm going to sort out some tracker so I can see exactly how you guys got here.
2005-11-16 mech-and-cow
I've submitted a diff to the cowsay maintainer because one of the templates was not working. It's probably an abandoned package by now, I don't know. I don't expect much back.
1,3c1,4 < ,-----. < | | < ,--| |-. --- > $the_cow = <<EOC; > $thoughts ,-----. > $thoughts | | > $thoughts ,--| |-. 11c12 < .==\ /_\ --- > .==\\ /_\\ 13,21c14,18 < (oo)\_______ /' / | | < (__)\ )\/\ /' / | `i < ||----w | ___,;`----'.___L_,-'`\__ < || || i_____;----\.____i""\____\ < < < < < --- > ($eyes)\\_______ /' / | | > (__)\ )\\/\\ /' / | `i > $tongue ||----w | ___,;`----'.___L_,-'`\\__ > || || i_____;----\\.____i""\\____\\ > EOC
Oh yeah, there's some really, really, really, really ... really funny mp3's out there, here's a wget for some tech-comedy:
for i in $( wget http://www.ampcast.com/music/22488/artist.php -O - -U Mozilla 2>/dev/null| grep mp3 | sed 's/.*href="//' | sed 's/".*//' ) ; do wget $i -U Mozilla ; done ;
(I've no idea why they are blocking wget, anyone who uses wget knows enough to change the agent ;)).
I have the go-ahead for opensouring the hotel booking page. There's now a link on the left for it. Feel free to download and send me updates, would be appreciated.
Had to run some huge, frightening bash script today to move a load of hosted domains in an emergency, thankfully it ran sound as a pound, had hardly any testing. People bang-on about RAD and how OOP languages like Perl, C#, java, python, haskell and how they can reduce development time, but to be honest, what I did could not have been done more efficiently with some high-level language in fewer lines. Perhaps perl would have come close, but why re-invent the wheel?
2005-11-15 waffle
Somedays are bad days, some days are good. Today was a mixture. Had wirtten 100 lines or so of bash to handle virtual domains moving from one box to another. It's quite a gotcha really - vpopmail has a passwd file for each virtual domain, the Maildir home can be ~vpopmail/domains(/[0-9]+)*/directory/name/. The numeric directory could be 0-n and chances are will not be the same between boxes. So my script has to work tomorrow, has to be able to process thees files, I've not had a huge amount of warning with this. Most of it has to be coded with my eyes closed as I can't be sure what it's going to do without having an identical system to test it on.
Lugradio is back! If you've not heard of lugradio its a rahdeo (intentional use of brommie accent) station. This is distributed in ogg and mp3 format via bittorrent, at some shocking speeds (average of 200kbyte/sec). The archives for seasons 1-3 are available via their website which provides endless amusement even if you don't generally like Linux. For those who don't know what a LUG is, it's a Linux User Group.
Theres a couple of funny pages I've found (yeah mentioned in lugradio):
- http://www.amibiosornot.com/
AMI BIOS or not, does this board have an ami bios chip? - http://geekz.co.uk/lovesraymond/
Everybody loves Raymond! This is a hilarious cartoon, reflecting the articles in ESR's very own blog.
Back in April-June some time I wrote a PHP shopping cart, designed for a Hotel-style booking, not interesting I hear you say? Well, I might go and open source it, I've been paid for it, and the guy who paid me kninda knows that I want to release it to the wild. I think I will, although I'll probably get no thanks for doing this. I'll come back to this sometime, I wouldn't want to release this if if were to offend anyone.
2005-11-12 cowsay
______________________________________/ cowsay me working on saturday in the \ | office. me go to gym now. me do | \ weights. / -------------------------------------- \ ^__^ \ (oo)\_______ (__)\ )\/\ ||----w | || ||
2005-11-10 sha1 bugs
This is really a warning to anyone who wishes to implement sha1_hmac into their PHP. For example, this following code has a couple of errors in it, http://cvs.drupal.org/viewcvs/drupal/contributions/modules/foaf/Attic/sha1lib.inc?rev=1.2. These are not apparent unless you look for them. There's no point detailing them here since there are other implementations. What is disturbing is that the code could never have been properly tested by the author otherwise they would have noted it's imperfections. For the time being I would suggest that people wishing to use sha1_hmac that they use PHP OpenID library functions. Seems safer at least for the moment. There are sha1 functions in PHP, but not a hmac.
2005-11-08 superserver
I've been making a superserver at work, there's a process that we need to keep running post-boot, it's one of those processes that has to run in foreground, it's not a service (obviously), and so it has to attach to sit in the taskbar someplace. Oh and it has a mainwindow. I've no idea how to get past the tough bit - attaching it to the main console. Seems in c# you can pick out the mainwindowhandle, but you can't write to this value, otherwise rdesktop users could send a process to the main console, where it does not belong. But this just sucks, why can't the mainwindowhandle be nulled? I'm happy with the process running in it's own container some place, it can be killed if required.
2005-11-07 more moan mono
Mono seems to be ok. I seem able to do stuff in mono with better efficiency than in .net. I really don't know what the psychology is, but the stuff I make in mono is tidy and clean. Stuff I do in .net is bloated. That aside, I was puzzling over some code that would normally be solved simply. However in c# you have to put a lock onto an object rather than just delcare a method as synchronised. I suppose this could have some advantage if you wish to use a method on certain objects. However, that could possibly be a bad design, why not segment the code so that those individual methods have their own synchronisation?
import java.io.*;
public class Synchron implements Runnable
{
public Synchron()
{
System.out.println( "Test" );
for( int i = 0 ; i<10 ; i++ )
{
Thread t = new Thread( this );
t.start();
}
}
public synchronized void syncd()
{
System.out.println( "I'm in the event ");
try
{
Thread.sleep( 1000 );
}
catch( InterruptedException ie )
{
}
}
public void run()
{
System.out.println( "123" );
syncd();
}
public static void main( String[] args )
{
Synchron s = new Synchron();
}
}
and in c#
using System;
using System.Threading;
public class test
{
public test()
{
Console.WriteLine( "Test" );
for( int i=0 ;i<10 ; i++ )
{
Thread t = new Thread( new ThreadStart( function ) );
t.Start();
}
}
public void syncd()
{
lock( this )
{
Console.WriteLine( "I'm in the event " );
System.Threading.Thread.Sleep( 1000 );
}
}
public void function()
{
Console.WriteLine( "123" );
syncd();
}
static void Main()
{
new test();
}
}
The resulting output should be the same from both applications.
123 I'm in the event 123 123 123 123 123 123 123 123 123 I'm in the event I'm in the event I'm in the event I'm in the event I'm in the event I'm in the event I'm in the event I'm in the event I'm in the event
I've also made some changes to the playlist PHP code... There's now a format option. playlist.php?format=m3u I'm going to add other formats to this too, it's not that complex, I've kept it simple, if you like, send me a diff ;). Source code is available here: playlist.phps.
2005-11-02 openbsd 3.8
OpenBSD 3.8 was released yesterday, it's on all the mirrors already. I've just burnt the cd, heres the copy/paste for it. Please get their posters/cds, the 3.5 cd has what I think is the best inlay so far.
mkdir OpenBSD mkdir OpenBSD/3.8 mkdir OpenBSD/3.8/i386 cd OpenBSD/3.8/i386/ wget -c ftp://ftp.jyu.fi/pub/OpenBSD/3.8/i386/* cd ../../.. mkisofs -r -b 3.8/i386/cdrom38.fs -c "boot.catalog" -o ~/cd.iso OpenBSD cdrecord dev=ATAPI:0,0,0 ~/cd.iso
I'm currently working on a simple page for people to register DNS entries on the domains that I have. There are no restrictions as yet to naming etc. This should be of interest to someone who does not want to register a name at cost. Once complete the pages will be available for download etc.