IIS and Tomcat … no end of problems.

Ok … I give.  Uncle.  I am through working on this for now, it is far too much wasting of time.

I have been trying, since Friday, to get IIS working with either Apache
(on the same box) or Tomcat (via the JK connectors).  I am able to
get Apache using port 81 to work, however now I have to deal with the
non-standard port issues.  So I found this nifty
isapi_redirect.dll that is supposed to allow me to connect IIS to
Tomcat directly!  No Apache required …

Yeah … right!

I have now spent almost eight additional hours working on this, and
have got nowhere.  The code, installation, and configuration are
simply poorly documented and do not work as described.  On top of
this, no matter what I do, I am unable to get *any* form of logging out
of the isapi_redirector.  I have been hand-hacking the registry,
and it seems to be working … to a point.  The IIS web logs
indicate that the defined URIs are being sent to the .dll … however
all I get are 404 not found errors.

I have now tried both the JK and JK2 versions of this thing … and
they both behave in a similar manner.  I’m stuck for now, and hope
that some day I am able to uncover the deep mystery of why this stupid
thing just won’t work.

I’m somewhat comforted by the posts that I have found all over the web
that seem to express the same feelings … no support, poor
documentation, inconsistenf operation.  It’s too bad … it would
be a nice solution for me!

Apache and IIS on Windows Server 2003, Part 2

I made some progress, and learned more about installing both of these
applications on Windows Server 2003.  First, if you use the
command “netstat -an” you can see the ports that are in use.  I
actually knew about netstat, however it was the “-an” option that
revealed the core information.

That piece of information was that both Apache and IIS both claim port
80 – in its entirety – across all IP addresses … no matter what the
settings in the various configuration files lead you to believe. 
Upon reading the documentation about Apache on Windows,  there is a note that is not completely clear:

Because Apache cannot share the same port with another TCP/IP
application, you may need to stop, uninstall or reconfigure certain other
services before running Apache. These conflicting services include other WWW
servers and some firewall implementations.

I would not have believe that this is the case even on different IP addresses. 
But this is the case.  So there is no way (that I can find) to
install both of these products, on different IP addresses, on the same
machine.  Done.

Ok, so then what is the solution?  I have now found two ways to
resolve my situation.  First, I experimented with creating a
virtual server in IIS that is simply a redirect to Apache on a
different port.  So I installed Apache on port 10.0.0.5:81, and
then configured IIS on 10.0.0.5:80 with a redirect to
10.0.0.5:81.  This worked!

The second solution is even better for my purposes.  I was
actually installing this to get a “JAM” application … Java, Apache,
MySQL … installed.  The actual configuration is for Apache to be
the web server, it uses mod_jk to connect to Tomcat, and the Tomcat is the Java container.  The Java application is what accesses MySQL through JDBC.

As I was reading on the configuration of  Tomcat I found that
there is an IIS version of mod_jk!  The mod_jk
isapi_redirector.dll will allow me to directly connect IIS to Tomcat
without requiring Apache … duh!  I should have figured that
someone would have written such a connector.

I’ll update the progress on Monday or Tuesday … I downloaded the
components, but ran out of time to get this new method installed and
working.

Wells Fargo On-Line Banking

I have to admit that I am impressed – over and over again – by Wells
Fargo and their On-Line Banking. When my old bank – First
Interstate – was acquired, I immediately joined the new Wells Fargo
On-Line Banking program, and was impressed that I could manage my
personal and business accounts in a single application. The user
interface was easy to use, and I even was able to mix personal and
business on-line bill paying in an efficient manner.

A while back, they further impressed me when I had to locate some old
bank statements for one of my companies. Not only did they allow
my to locate them on-line, they provided them in Adobe .PDF format …
for NO charge! I was able to download and print the statements that I required to complete my taxes …

Today … more. When I happened to go and look at one of my
business accounts, I noticed they added a new link next to each check
that has cleared – View. Yes … when I click the link, it
provides the scanned image – front and back – of the check
itself! Oh yeah … and it appears to be free … no additional
charge!

I’m impressed with Wells Fargo On-Line Banking …

Open Source BPEL Engine

While working on our Web Services Substrate, we did a lot of looking at
the coming BPEL standard.  While it doesn’t do everything that we
wanted, it provides some powerful functionality, and will only evolve
to be better and better.  I happened to stumble on this Open
Source project providing a BPEL engine … I’m going to go take a look
at it …

ActiveBPEL 0.9.2. An engine that executes BPEL processes. [freshmeat.net]

Problems with PayPal IPN

I am slowly learning about the issues with relying on an Internet
service like PayPal. They have somehow totally screwed up their
software, and it is no longer calling our system to notify us of
payments as of today.

We use PayPal to allow our 80211.net
Internet customers to purchase Internet Access on-line. When they
make a payment, they expect to be able to use the Internet
immediately. This has been working great up until now, using the
PayPal IPN (Instant Payment Notification) service. Somehow PayPal
has screwed up their systems, and the IPN notifications are no longer
hitting our server … so people are paying, and their accounts are not
being updated. So today I have been fielding calls and having to
manually credit people’s accounts … which sucks.

This is just another example of where the concepts around the Internet
and hosted services breaks down. It is really too bad that PayPal
didn’t test their software updates better before deploying them on
their “live” site. People complain about Microsoft and Windows
… but this is a more important example to me since it is costing me
time and money … and there is no one that is going to recover the
costs for me.

Novell NetDrive … a dying product?

I have been using NetDrive
(http://support.novell.com/servlet/filedownload/uns/pub/ndrv41862.exe/)
for years now, and it is a very innovative piece of software. It
completely alters the way that people use FTP to transfer files …
making it as easy as “mapping a drive”. With NetDrive I can “map”
a drive letter, say “N:” to my FTP server on the Internet. I can
then “drag and drop” files just like any other drive on my system.

The real issue with FTP is that it is not the most secure protocol that
you can use. Most Linux and UNIX users are using SSH and SCP
instead. SSH is the “secure shell” and combined with SCP, a
“secure copy” it allows you to access your remote boxes through an
encrypted connection. I use both of these all day, and what hit
me was that the usability of SCP – even using WinSCP – is not equal to
that of NetDrive.

I started to check and see if Novell had released a version of NetDrive
that would use the secure protocols, and found that I could not locate
any newer versions of NetDrive! The last one I can see if from 17
Apr 2003! And there is no apparent work on a version that
supports SSH/SCP …

Here they have a very powerful tool that could be used to “seed” the
market and alter how people access Linux from Windows … branded with
Novell’s name … and they seem to be letting it die.

Novell … drop it into Open Source … or update it! You are again allowing a valuable beachhead to disappear …

Issues with RSS …

This article addresses an interesting dilemma … the constant
“polling” required by RSS news feed readers.  RSS works by having
the “reader” application poll the RSS file to see what new articles, or
items, are present.  As described, the RSS reader applications
often use the “top of the hour” as the time for when they fetch the
file to check for new items.

One thing that can be done on the server side is to insert caching
reverse proxies that are designed to scale these types of
requests.  When I was working at Novell, we used the BorderManager
product for exactly this type of scaling.  It was able to handle
hundreds of thousands of simultaneous TCP connections, and would serve
up the cached content … in this case a simple RSS XML file.

There are some ways that this situation could be improved … but it
would take coordination of the various RSS news reader developers …
and possibly more evolution of the RSS “standard” …

When RSS Traffic Looks Like a DDoS [Slashdot:]

80211.net and Paypal …

The last several days I’ve been completing something that I wanted to
do for over a year … integrate credit card processing into our
wireless network.  My friend and I run a Wireless ISP called 80211.net and we have several Internet Cafe’s and a local residential network.  We have been running NoCatAuth as the authentication and authorization system.

Several years ago, I contributed to the project by updating the LDAP
connector so that we could use an LDAP Directory as the primary user
database.  I picked this direction since we could use the ‘account
expiration’ features of the directory and allow users to purchase time
on the network.  I also designed and implemented a ‘cash Internet
Access coupon’ system so that the Cafe’s could sell the coupons to
customers.  The user entered the coupon information and the system
would credit their account by incrementing the ‘account expiration’
date.

The coupons have been working great for years, and offer a variety of
benefits to our company, and the customers.  But we wanted to
implement accepting credit cards to augment the coupons, and to allow
users to buy Internet Access when there was no one present.

We picked Paypal as the first
solution for this.  And I am almost complete with the work. 
It’s been fun learning how this works.  I had to write some new
scripts and web pages … first to offer the credit card payment, then
to verify the user account, and then to redirect the user to Paypal …
passing some custom transaction information.  Paypal has a very
simple system for passing these variables, and then offers their IPN
(Instant Payment Notification) that will call the script on my server
that adds credit to the users account.

Lastly, upon completion, Paypal will redirect the user to another one
of my scripts so that I can provide details of the transaction.  I
am now working on the last two steps of this process, and hope to
complete it in the next couple of days.  My folks are coming into
town today, and so I’m not sure how much time I’ll be able to spend
working on this.

Migrating NetWare to Linux …

Today I took another step forward in migrating my last NetWare servers to Linux.

While working at Novell I did a considerable amount of R&D on my
own time to set up a complete ISP using Novell’s NetWare. I
experimented with web servers, mail servers, IRC servers, voice/video
conferencing servers and even wrote an MP3 streaming server for
NetWare. By the time that Eric Schmidt joined us at Novell, I had
already created a long list of limitations that were stopping me from
being successful. The irony of the entire situation is that now NetWare
v6.x really has some powerful Internet services … but it’s really too
late. SuSE Linux is going to be the future for Novell.

One of the primary Internet services – of course – is the web server. As a part
of the executive team that partnered with Netscape to create the
Novonyx venture, I quickly embraced the Novonyx SuiteSpot Web Server
on NetWare and have been running it since. To facilitate my move to Linux, I
wanted to first move to Apache on NetWare, and then take the configuration to Linux. Once I have Apache on NetWare working
solid, then changing out the kernel underneath is no big issue.

To get from Netscape Enterprise Server to Apache, the first thing that I did was to take copies of the two core
configuration files – obj.conf for Netscape, and httpd.conf for Apache.
What I found is that there were three core areas that I had to address:

  1. virtual host definitions
  2. log file specs
  3. cgi handling

The first two items in this list were fairly simple to convert. It really
just came down to creating all of the httpd.conf virtual host
definitions. In the obj.conf it might look like this:

<Client urlhost=”www.inevitable.org”>
   NameTrans fn=”document-root” root=”/users/org/Inevitable/Web/Docs”
</Client>

and so in the httpd.conf it now looks like this:

# the.inevitable.org
<VirtualHost 63.230.21.106:80>
    ServerAdmin support@onepostal.com
    DocumentRoot /users/org/Inevitable/Web/Docs
    ServerName the.inevitable.org
    ErrorLog /users/org/Inevitable/Web/Logs/error_log
    CustomLog /users/org/Inevitable/Web/Logs/WebHits.log combined
</VirtualHost>

Note that this also addresses the ‘combined’ log file format and location. In the obj.conf I had to have some lines like:

Init
log0=”/users/org/Inevitable/Web/Logs/WebHits.log”
format.log0=”%Ses->client.ip% – %Req->vars.auth-user% [%SYSDATE%]
“%Req->reqpb.clf-request%” %Req->srvhdrs.clf-status%
%Req->srvhdrs.content-length% “%Req->headers.referer%”
“%Req->headers.user-agent%”” fn=”flex-init”

and:

<Object ppath=”/users/org/Inevitable/Web/Docs/*”>
    AddLog fn=”flex-log” name=”log0″
</Object>

… to write to the logs. Apache is much simpler. It’s just the line within the virtual host above:

CustomLog /users/org/Inevitable/Web/Logs/WebHits.log combined

With all of my .conf file converted, everything seemed to work well. Except cgi …

NetWare was always an interesting animal to deal with when it comes
to server-side development languages. There are a lot of
intersting hoops that have to be jumped through to invoke scripts …
and most of this is done through “NLMs” which are the binary
executables on NetWare. On my servers I had some scripts written
in various languages – including Netbasic and Novell Script – and this
turned out to be the toughest part to define and get working.

The first step was to get ‘mod_lcgi.nlm’ working. I added it to the httpd.conf with the following line:

LoadModule lcgi_module modules/mod_lcgi.nlm

I then had to find the right combination of statements to
get mod_lcgi working properly. The following section I added to
the end of the httpd.conf … and it took me a *long* time to get it
right!

<IfModule mod_lcgi.c>
    AddHandler lcgi-script .nlm .ns .asp .nsp .pl .bas

    ScriptAlias /nsn sys:/nsn/web
    LCGIModuleMap sys:/nsn/lcgi/cgi2ucs.nlm /nsn .ns

    ScriptAlias /sp sys:/nsn/web
    LCGIModuleMap sys:/nsn/lcgi/scrptpgs.nlm .asp .nsp /sp

    ScriptAlias /perl sys:/perl/web
    LCGIModuleMap sys:/perl/lcgi/cgi2perl.nlm .pl /perl
    AddEnvVar PERL_ROOT sys:/perl/web

    ScriptAlias /netbasic sys:/netbasic/web
    LCGIModuleMap sys:/netbasic/lcgi/cgi2nmx.nlm /netbasic .bas
</IfModule>

Once I had that working, I just had to add the mod_lcgi.nlm to the
apachemodules directory … uh and then things went very wrong.

When I restarted the server, mod_lcgi barfed with a whole lot of
“Unresolved Externals” … symbols that could not be resolved and
dynamically linked. It appeared that mod_lcgi.nlm was attempting to
autoload ‘nslcgi.nlm’ and dynamically link.  I had to dig for a
while to determine what
the issue was. I found this Novell TID
that explained my problem! I went and looked at one of my newer
servers where Apache was running fine, and sure enough I found an
updated ‘nslcgi.nlm’ in the
system directory of that server … once I updated this NLM I was
up and going …

I am now running all of my web sites on Apache on NetWare … and so it
is now just one next step to get all of this running on Linux … maybe
next week … 😉

If anyone else is trying to do something like this … feel free to contact me and ask questions!

Ken Novak, Bloglet, and virtualization …

I was in the middle of debugging my problems with Radio categories last night, when in walked Ken Novak.  I met Ken Novak a number of
years ago at the Foresight Institute conference. He is always
involved in some interesting and exciting areas … I always like to
hear about his work. Last night as we talked about our use and
experiences with RSS, Radio, and various RSS related companies and
products. I have been looking at an idea that I described to Ken
and he pointed me to Bloglet … my idea already exists! This is a very cool idea … a variation on what News Gator is doing … converting RSS feeds into e-mail messages.

We also spent some time laughing about the current state of Userland’s
Radio … it is becoming clear that most experienced users just plain do not mess with it
once it is working well for them.  I’m going to keep pushing the
limits to get my categories going … and I’ll keep backing things up
on the way.

We also talked about virtualization technologies … his interest and
recognition of the value, and my experiences with Virtual PC
2004.  We talked about how we both expect to see people begin to
trade in Virtual Hard Disks … the large files created by Virtual PC
that contain the hard disk information.  We both see that this
goes another step beyond the current file-trading that people are doing
today.

I had hoped he would be here the rest of the weekend … we’ll have to catch up on-line …