Last weekend, we ran into a mystery problem with the Domino 8.5b1 server. For several weeks, we had had no problem sending mail from Thunderbird through the newly-upgraded server. Everything worked the way it had for months with 8.0b3.
However, we replaced the old Linksys WRT54G rev 5 with an Apple Airport Extreme, the 802.11n model. Set up the port forwarding to match what the old Linksys had used, and everything seemed OK until we tried sending email out from Thunderbird on machines connected to the local network. Everything was rejected at connect time saying the server was rejecting relay for policy reasons.
I noticed a difference at the console: with the old Linksys, when you sent mail, the server reported that the connecting SMTP host was your actual IP address on the local area network. With the Apple, the router itself (192.168.1.1) was reported as the connecting host, and it rejected it consistently. No amount of fiddling with the DMZ settings (what Apple calls the "default host," rather than the DMZ) or port forwarding would change this.
OK, let's have a look at this. We tried changing the port forwarding, no help. Fiddled with some of the SMTP restrictions, and found a major bug in 8.5b1. I mean, a MAJOR bug. One of the settings I tried was to set up a user group including all of our Notes names, and put that in the Outbound Restrictions setting of the server config document, saying, "anyone in this group is allowed to relay." That didn't work, mostly because I suspect it only checked the Notes names, not our SMTP names. I then checked the documentation and it said that we could also list individual SMTP addresses in that field to have the server permit those specific SMTP addresses to relay.
I put the field names in the field, killed SMTP, restarted it, and... the server exploded.
Well, not literally. But Domino immediately and consistently NSD'd.
Take those SMTP addresses out, and the server would start again.
I sat and thought about it for a couple of days, and then tonight decided to fool with it again.
This time, I changed some settings in the inbound SMTP restrictions. In there, for months, we had had a setting saying that the only allowable external domain one could route to was myrabbits.org, which is Nora's domain. 8.0 thought of this as an external domain even though another server config document clearly told it that myrabbits.org was hosted on that same box. I removed that, and after checking to make sure that all external hosts were checked for relay but that all authenticated users could relay, I restarted SMTP.
Not only did the server not crash, mail actually routed. I even went outside and used the Blackberry to send some email from my weightlessdog.com account to an external address, and it went with no fuss. The server appears to still be blocking out real spammers, so it appears that things have been healed.
Just like the thing the other day with the Notes 8.0x client on XP at work, I don't know why it broke, but it's workin' now.
I owe Phigment some beerz. We should organize a DC/Baltimore area events sometime.