Apparently there's a way you can get your computer to hook into the worlds financial systems and automatically buy/sell shares. How much attention did you pay in AI class?
Makes me think of the day when all the stock markets will be run and controlled by robots... can't wait
Monday, 6 April 2009
Thursday, 2 April 2009
Another Day, antoher Crisis
Just woke up this morning and thought I'd get working on tool I'm creating to be able to monitor if and when we get put onto blacklists, when lo and behold, I discover our default ip is again on the cbl blacklist. A couple of hours of diagnosing and debugging later and I've found that our hosting provider as a part of their recent upgrades decided to go and change our hostname settings on our MTA, causing everything to come off of 'localhost.localdomain'. Wonderful. As if we didn't have enough problems, now we've got a MTA that's misconfigured and for the changes to take effect it needs a nice reboot. Well, luckily the reboot went well and our MTA is again sending happily off of a proper hostname.
Friday, 27 March 2009
Lack of Planning
With this job at Jericho, it feels like I'm just constantly putting out fires. In one way that's a good thing, meaning that at least I'm busy and have work to do (which is not a bad thing in a recession). However, it does mean that I find less time to sit down and do some real planning.
For example, I've started looking into getting some industry qualifications in order to put into practice my idea that in the IT industry, to be successful, you must be constantly learning. Some industry qualifications would also look very nice on my resume. The ones I started looking at have been the Red Hat certifications and the Cisco certifications, although I wouldn't be opposed to getting Microsoft Certified, but that's a low priority. You can find so many '.NET' monkeys around that it would almost be an advantage (in my humble opinion) to not have any certs from the software giant.
Also, I've been thinking about starting my own Linux consulting firm. Nothing big, mostly just a way for me to make some extra cash on the weekends and further refine my skills. I think there's a real gap in the market here in NZ. I was also inspired by learning about OpenLogic and seeing one of their presentations. This was a very nice reminder that FOSS is in fact used in large enterprise, a fact that I sometimes forget working in a .NET shop like Jericho.
Here at work, the jobs keep piling up. On the system administration side of things I have to set up a box to act as a router/firewall/logger etc... For this I'ver gone with using SmoothWall, which is a dedicated 'router' distro which I've used before with success at Primesoft. The job after that will be to set up our own email server, whose main job will be to replace the contract we have with Net24/IPX. The machine's main purpose will be to act as a spam filter and a proxy. Although, I'll also get it to keep copies of the mail locally and set it up with a IMAP and a POP server, just in case our local Exchange server dies, so we have access to our mail. I'm probably going to go with setting up a Ubuntu box with Postfix and SpamAssassin. I also really want to set up Bayesian filtering on the server. The tricky part there's going to involve setting up a feedback mechanism, so that we can get the employees to train the filter.
On the Deliverability side of things, the job we're doing right now is creating a classifier to classify our asynchronous bounces. Previously we've been using a product called ListNanny, but it has failed us miserably. Obviously it's a product which is made for small lists but stands no chance when it comes to enterprise class ESP's. So far, we've got a set of regular expressions which we use to determine the classification. The set is not large enough and running a test over the last 10,000 messages only yields about 20-30% match rate. So, we're not only going to have to increase the size of the regex set, but also create a system to maintain such a set. There is also the idea I had to use that bounce classifier that I found in CPAN, but when I tested it out, it just didn't seem that accurate. There is a possibility to rewrite/redifine it seeing as how it is open source, but having our own solution has it's own advantages.
Personally, due to the ever changing nature of asynchronous bounce messages in my opinion, this would be a perfect problem to tackle using an AI technique such as a Bayesian classifier, the same as is used by SpamAssassin. Thinking about it, it wouldn't require that much tweaking. Simply, instead of getting the system to give a message a spam score, you give a score for each category (reply, hard bounce, soft bounce, complaint ....). This system would also need to have a feedback mechanism, to keep the false positives and false negatives to a minimum. This would actually be a really boring job, effectively resigning some poor soul to being a 'bounce monkey'.
The other project we've got going is trying to create a real time 'whiz-bang' graphing application for monitoring our sends. Something similar to glTail. I've already started looking at employing the same approach as glTail i.e. a ssh connection that is kept alive providing the 'stream' which is then used to create the numbers to be displayed on the graph. I've started looking at using either Perl (for its easy text manipulation) or Java (for its cross platform-ness). The problem with Java is that it doesn't have any official ssh libraries. There are libraries out there, but I have my doubts as to the quality. With Perl, it would only run on Linux so integrating it with ssh would be a piece of cake (well, actually a pipe, but hey...). I guess I could try maybe using cygwin and integrating that with a java front end somehow.... Anway, it'll probably be a while until I have enough time to start worrying about that.
For example, I've started looking into getting some industry qualifications in order to put into practice my idea that in the IT industry, to be successful, you must be constantly learning. Some industry qualifications would also look very nice on my resume. The ones I started looking at have been the Red Hat certifications and the Cisco certifications, although I wouldn't be opposed to getting Microsoft Certified, but that's a low priority. You can find so many '.NET' monkeys around that it would almost be an advantage (in my humble opinion) to not have any certs from the software giant.
Also, I've been thinking about starting my own Linux consulting firm. Nothing big, mostly just a way for me to make some extra cash on the weekends and further refine my skills. I think there's a real gap in the market here in NZ. I was also inspired by learning about OpenLogic and seeing one of their presentations. This was a very nice reminder that FOSS is in fact used in large enterprise, a fact that I sometimes forget working in a .NET shop like Jericho.
Here at work, the jobs keep piling up. On the system administration side of things I have to set up a box to act as a router/firewall/logger etc... For this I'ver gone with using SmoothWall, which is a dedicated 'router' distro which I've used before with success at Primesoft. The job after that will be to set up our own email server, whose main job will be to replace the contract we have with Net24/IPX. The machine's main purpose will be to act as a spam filter and a proxy. Although, I'll also get it to keep copies of the mail locally and set it up with a IMAP and a POP server, just in case our local Exchange server dies, so we have access to our mail. I'm probably going to go with setting up a Ubuntu box with Postfix and SpamAssassin. I also really want to set up Bayesian filtering on the server. The tricky part there's going to involve setting up a feedback mechanism, so that we can get the employees to train the filter.
On the Deliverability side of things, the job we're doing right now is creating a classifier to classify our asynchronous bounces. Previously we've been using a product called ListNanny, but it has failed us miserably. Obviously it's a product which is made for small lists but stands no chance when it comes to enterprise class ESP's. So far, we've got a set of regular expressions which we use to determine the classification. The set is not large enough and running a test over the last 10,000 messages only yields about 20-30% match rate. So, we're not only going to have to increase the size of the regex set, but also create a system to maintain such a set. There is also the idea I had to use that bounce classifier that I found in CPAN, but when I tested it out, it just didn't seem that accurate. There is a possibility to rewrite/redifine it seeing as how it is open source, but having our own solution has it's own advantages.
Personally, due to the ever changing nature of asynchronous bounce messages in my opinion, this would be a perfect problem to tackle using an AI technique such as a Bayesian classifier, the same as is used by SpamAssassin. Thinking about it, it wouldn't require that much tweaking. Simply, instead of getting the system to give a message a spam score, you give a score for each category (reply, hard bounce, soft bounce, complaint ....). This system would also need to have a feedback mechanism, to keep the false positives and false negatives to a minimum. This would actually be a really boring job, effectively resigning some poor soul to being a 'bounce monkey'.
The other project we've got going is trying to create a real time 'whiz-bang' graphing application for monitoring our sends. Something similar to glTail. I've already started looking at employing the same approach as glTail i.e. a ssh connection that is kept alive providing the 'stream' which is then used to create the numbers to be displayed on the graph. I've started looking at using either Perl (for its easy text manipulation) or Java (for its cross platform-ness). The problem with Java is that it doesn't have any official ssh libraries. There are libraries out there, but I have my doubts as to the quality. With Perl, it would only run on Linux so integrating it with ssh would be a piece of cake (well, actually a pipe, but hey...). I guess I could try maybe using cygwin and integrating that with a java front end somehow.... Anway, it'll probably be a while until I have enough time to start worrying about that.
Tuesday, 24 March 2009
Long Overdue Update
It's been a while since the last time I wrote. It's been about 4 months since I started at Jericho Ltd and the job now takes up a majority of my time. Everything from producing deliverability reports for the clients to making sure our network is secure is handled by myself. I find myself often looking at the huge list of things to do and getting demotivated. It would take another 4 months just to get everything off the list of things to do.
On a brighter note, I went to a 'Agile Professionals' conference (agile as in the methodology) today. The guy (Jeff Smith from Suncorp) didn't have a bad thing to say about open source which was good. He also talked about the necessary attitudes and philosophies that you should have when trying to create teams (people centric) and where the innovation in a company should come from (bottom up). These were all very good points, it was a shame that only myself and Clint showed up to the conference, it would have been nice to have a few more people from work there.
On a brighter note, I went to a 'Agile Professionals' conference (agile as in the methodology) today. The guy (Jeff Smith from Suncorp) didn't have a bad thing to say about open source which was good. He also talked about the necessary attitudes and philosophies that you should have when trying to create teams (people centric) and where the innovation in a company should come from (bottom up). These were all very good points, it was a shame that only myself and Clint showed up to the conference, it would have been nice to have a few more people from work there.
I've also been reading. Finished reading Fred Brooks legendary book 'The Mythical Man Month' and the first two books in John Scalzi's Old Man's War series. The mythical man month is a real gem and deserves its reputation as a great book. I don't think I've ever seen so many truisms about software expressed so eloquently. John Scalzi's work is good as well, although the claims on the cover of the book about him being Heinlein's equal are exaggerations.
Friday, 12 December 2008
Learning Perl
I'm currently reading the book 'Learning Perl' and I have to say its a really well written book. It teaches the Perl language superbly and the exercises at the end of each chapter are detailed enough to give you a chance to try out the concepts learned in the chapter while at the same time being succinct enough to keep the reader interested.
As far as Perl is concerned, I think I may have found my newest scripting language. For someone that does most of their programming in Java and having only tried Ruby briefly, Perl comes as a revelation. That a language can be so optimized, to enable the programmer to translate his thoughts as quickly as possible into code is a powerful concept.
I originally started learning Perl as a part of a project at work. We have to make a program to parse an email and classify it as a 'bounce' or a legitimate reply. Stephen, the guy I work with, wrote a parser from scratch in PHP. I'm hoping to make use of one of the Perl modules available, such as the 'Email' module I found after a quick google search.
The other project I was hoping to make use of Perl for was on my home file server. I have a lot of music files on there and I have a suspicion that a lot of space is wasted because of duplicates. I was thinking of writing a program to scan a directory and make a simple database of every file and its MD5 hash. Then comparing these hashes and giving a report on the suspected duplicate files. This seems like the kind of job that Perl would be perfect for.
As far as Perl is concerned, I think I may have found my newest scripting language. For someone that does most of their programming in Java and having only tried Ruby briefly, Perl comes as a revelation. That a language can be so optimized, to enable the programmer to translate his thoughts as quickly as possible into code is a powerful concept.
I originally started learning Perl as a part of a project at work. We have to make a program to parse an email and classify it as a 'bounce' or a legitimate reply. Stephen, the guy I work with, wrote a parser from scratch in PHP. I'm hoping to make use of one of the Perl modules available, such as the 'Email' module I found after a quick google search.
The other project I was hoping to make use of Perl for was on my home file server. I have a lot of music files on there and I have a suspicion that a lot of space is wasted because of duplicates. I was thinking of writing a program to scan a directory and make a simple database of every file and its MD5 hash. Then comparing these hashes and giving a report on the suspected duplicate files. This seems like the kind of job that Perl would be perfect for.
Thursday, 13 November 2008
Asterisk web GUI doesn't work
After the last post where I thought asterisk was pretty much set up, I was really disappointed to find I couldn't for the life of me get the web gui to work properly. By this time however, I knew all about how to get the zaptel driver configured and running properly, so we went back to the machine we set up following the guide here. After restoring the backup I found that the driver didn't work properly, but that was fixed by just compiling and installing it again. Then going into the asterisk command line, we still didn't have the 'zap' commands. After searching around I finally found that it was due to the asterisk people renaming everything to do with zaptel to DAHDI (Digium Asterisk Hardware Device Interface). So now that we got that working its just a question of getting the correct configurations and the correct rules set up. So, hopefully another day or two should sort it.
Tuesday, 11 November 2008
Asterisk on a debian box
The job I'm currently doing at the moment is trying to set up a VoIP/PBX box which sits on the network and basically routes all local calls through the PSTN and routes all international calls through a VoIP provider (kind of like Skype). So the idea is that you save money on international calls by not having to use the telco's network.
So the idea was pretty good, and it seemed that the Asterisk software was pretty mature, but I did run into some problems. The first time I tried setting this up I tried downloading, compiling and installing the zaptel driver and the asterisk software manually. This led to some kind of error where no matter what happened Asterisk could not see the telephone cards. Although, for some reason the web interface working on top of Asterisk could.
But anyway, the way I finally got it working was to download and install the zaptel source code for the drivers from the debian repositories and then compile the driver. Apparently you can't get pre-compiled drivers from the debian repo's. The package itself (zaptel-source) had a dependency on another package called 'module-assistant'. After installing the zaptel-source package through apt, I compiled and installed the package using the command 'm-a a-i zaptel' i.e. 'module-assistant auto-install zaptel'. This managed to install the drivers properly. Then, I installed the asterisk version in the debian repository as well by just running 'apt-get install asterisk'.
Tomorrow, I start configuring the asterisk installation.
So the idea was pretty good, and it seemed that the Asterisk software was pretty mature, but I did run into some problems. The first time I tried setting this up I tried downloading, compiling and installing the zaptel driver and the asterisk software manually. This led to some kind of error where no matter what happened Asterisk could not see the telephone cards. Although, for some reason the web interface working on top of Asterisk could.
But anyway, the way I finally got it working was to download and install the zaptel source code for the drivers from the debian repositories and then compile the driver. Apparently you can't get pre-compiled drivers from the debian repo's. The package itself (zaptel-source) had a dependency on another package called 'module-assistant'. After installing the zaptel-source package through apt, I compiled and installed the package using the command 'm-a a-i zaptel' i.e. 'module-assistant auto-install zaptel'. This managed to install the drivers properly. Then, I installed the asterisk version in the debian repository as well by just running 'apt-get install asterisk'.
Tomorrow, I start configuring the asterisk installation.
Subscribe to:
Posts (Atom)