Recently in Services Category
File space usage has grown significantly since our last server upgrade. I expected the current disk space to hold us for 4 years, which is about what I budget for the overall hardware. Alas, it is looking like perhaps I should upgrade the storage sooner.
The fun part is, gigo.com currently is comprised of 5 disks, but realistically only 250 gig usable space to end users. We spend a fair bit on redundancy, in case of catastrophy. Here is how it is broken out:
- disks 1+2: main system, this is where the main files are stored, and served from. Anything we do happens here. This is continously mirrored, so that if either disk fails, the system can quickly recover and keep running. And, I can put in a replacement to restore redundancy "hot".
- disks 3+4+5: Backups. At any given time, 2 disks are hot and mirrored; and 1 is cold (offsite, my house). Periodically, I take the cold disk, stop at the colo, swap out one hot disk for the cold one. The server will resync the mirror, and the disk I have in my hand goes back home - with a copy of several days worth of our files. And, total time in the colo is <10 minutes to sign in and swap a disk.
With that in mind, if I do upgrade storage, I'm not upgrading just one disk, but realistically all 5. Ooof!
What I'm looking at doing is:
- 2 enterprise class SATA 1GB disks - $160 each + the governor's ransom - matched set for mirroring.
- 3 desktop class SATA 1.5GB disks - $120 each + the governor's ransom - matched set for mirroring.
The backups can be desktop class; they get hit with less work, don't need to be as fast, and we can afford a failure there without a serious panic. They should however be larger than the main system drives, since we backup multiple days worth of changes (currently we back up ~20 days worth of changes; this number varies based on space available and number of changes made in a day).
I'm looking to try and help raise about half this cost - so a target of $375. If you're a significant user of gigo.com and can help, please contact me. Lady Visa will be covering the gap; I'm aiming to do this hardware changeout over the xmas break.
If you maintain an "aliases" file on gigo.com for one or more of your domains, please note the following changes are being made:
I have set up a jabber server on gigo.com, for users of gigo.com.
If your web site and email both are located on gigo.com, you can log into jabber as username@domain.name using your regular email password. Only valid email addresses will work. You an use any of your working email aliases.
If your web site is not hosted here, but your email account is, you can use loginname@gigo.com with your gigo.com password.
Notes to self, and for anyone else who cares..
Update 1/1/2008: Contact me to enable your account on jabber. I'll copy your email password and enable your access.
Placeholder, will describe SquirrelMail and procmail filtering.
(primarilly for unix users)
I contribute resources to those in the SCA, or Society for Creative Anachronism. This is something I do where/when appreciated, quietly and mostly behind the scenes, but won't twist folks arms to do.
Contributing to gigo.com's funding
Updated 7/2/2007
Please see http://pal.gigo.com
Contributions towards operating this server are greatly appreciated. In the past people have directly contributed hardware in times of need, and at other times, funds. Our original estimate was that this could cover the costs until 2006, but, that was .. not taking into account taxes (!). As such, the funds raised in 2003 have been used up, and I am back to operating this server 100% out of pocket. I'm not complaining; but I don't mind assistance either :-).
Money collected here will be put specifically towards hardware and colocation feeds. The hardware has been purchased, and colocation started. If you're willing to help towards these costs, click the PayPal button. Send whatever you feel is appropriate; common values have range from $5 to $100.
If you'd prefer to send me a check or want to meet up in person, email me instead. I'll give you my postal address via email, or we can make other arrangements.
Updated 7/2/2007
Please see http://pal.gigo.com/contributors.htmlThe following is the list of folks who have contributed since 9/2003, when the call was put out to upgrade the server and the colocation service.
"; if ($c == $third) print "
- Marco Nicosia - provided the rackmount case for gigo.com 2001-2003; still in use today for the backup colo option
- Coastal Web - providing via barter and kindness our current colo facilities for the last 2001-2003; continual ongoing backup colo services; and a hard drive
- Dave Matson - aka, the Hard Drive Genie. Hard drives appear out of thin air when the need arises.
- the Suchters - Split my colo costs @ meer.net for a few months in 2001 when we lost our free hosting
- Ulf Zimmerman - provided colocation after I left my position at CalWeb until 2001, giving gigo.com a stable home on the net.
About gigo.com hosted services
GIGO.COM hosts various services for friends and hobby groups. Typically such hosting is not going to be billed for as long as it is for a worthy cause, and my costs for hosting it are not unreasonable.
Previously we took no business opportunities at all. Now that we are in a commercial colocation center, with all the benefits (and costs!) association with it, we are no longer turning down business web site hosting. Fees for this will be negotiated on
a case by case basis, with the primary concern being cost recovery. Non commercial sies will still be free.
What we do do, for our friends, and for hobby groups who need a home:
- Web hosting for low impact sites
- Web hosting for higher impact sites, if colo fees are planned for
and covered
- Mail hosting and forwarding, especially those suffering from big-name
ISP mail blues.
- Mailing list hosting for mail-based forums
- DNS hosting and management, both static and on-demand dynamic
gigo.com is willing to host mailboxes for people who are contributing in some fashion to the hobby community. Approval for this is done on a per-request basis.
Mail *is* filtered coming into gigo.com by my rulesets. I run them as I see fit. I do not check the *content* of where mail comes from but instead check to make sure they come from sane places. Any mail I refuse, has a error message saying contact postmaster@gigo.com if it is legitimate, and that will always reach me.
Because I do not check the *content* of mail, a lot of spam still gets through. This is despite my blocking 1/3rd of all incoming mail. Do not forward it to me. I get enough as it is. If you are a shell account user on my system, type "man procmail".
non-technical hobbyists
gigo.com doesn't give to just the technical hobby community. we are willing to host email for anyone who shows a genuine need for a stable mailbox address, who is active in the hobby community. For these people, mailbox access is available via web at mail.gigo.com. Mail is also available via pop and imap mail programs; however, technical support for these are limited.
unix geeks
I will give shell account access only if I know you personally. This is not something granted to just anyone. Chances are if you want a shell account, you should probably be running your own unix box.
That said, if I do grant you access to use my system, you have your choice of how to get your mail. You can either use pop/imap/web, or local mbox format. Can't have both. My public-facing pop/imap servers use a sealed mailbox format. "pine" is an excellent imap client if you want to read mail via shell but still have it accessible via GUI mail clients. Lastly, if you want, just create a .forward file, and forward your mail to the account-du-jour. IF you change providers, you can just edit your gigo.com forward file to the new provider.
If you currently have your mail stored available via unix, and you want it via pop/imap/web, or vice versa, mail postmaster@gigo.com and ask for it to be changed.
How we filter, why we filter
[top] [smtp auth] [my networks] [bounces] [whitelists and blacklists] [realtime blacklists] [dns resolution] [smtp greetings] [unauth pipelining] [greylisting] [dialup users] [isp sanity check] [Get off the filter]
All gigo.com mail is scrutinized at the SMTP level. We do not look at message contents (unless the end user specifically sets up content filters). gigo.com looks at the *envelope* of the delivery (much like how postal mail uses envelopes but the contents may be totally different of what was represented).
When looking at the envelope we look for several signs that the mail is less than genuine. Some of the common checks we use are listed below:
- Is the sender authenticated? If the sender has successfully logged in with SMTP AUTH (a way of doing logins over SMTP), then we will implicitly trust the mail. The other paranoia goes away. If you are a user that wants to send mail out via me, enable SMTP AUTH in your mail program, and life is much, much better. (Todo: How-to docs for common applications)
- Is the sender from one of the gigo.com networks? I do from time to time consider adding specific static IP's to the list of "my network", which also bypasses most checks and will allow you to relay, if it is appropriate.
- Is it a "bounce" message, but to several recipients? Quite simply, bounces never legimately do this.
- Is the sender or recipient in any of the gigo.com whitelists? Sometimes we pre-approve specific senders to send us mail from any location. Some recipients prefer we pre-approve all mail going to them, and leave any spam fighting (if any) left up to them. We're happy to comply, even if there is a twang of pity.
- Is the connecting host on any blacklists? We make use of several blacklists. If mail is rejected for any reason from those lists, the details will be in the refusal. Also in the refusal is who to contact (the postmaster, which is always whitelisted). As of the time of this writing, those RBL servers are: relays.ordb.org, sbl.spamhaus.org, proxies.relays.monkeys.com, opm.blitzed.org, dnsbl.njabl.org, blackholes.wirehub.net, list.dsbl.org.
- Sender's address, does it resolve? We look at the sender's address and make sure that there is a mail server on the internet that is publicly known to take mail for this address. If the domain is expired, or if the domain name is not configured right, or if there is a typo on the spelling of the domain name, the mail will be delayed (but not totally refused) by our server. If the problem is not corrected, the sender's ISP will ultimately give up trying.
- Sender's greeting, does it follow the specifications? Lots of spammer software doesn't. Sadly, neither does a lot of end-user software (using SMTP AUTH bypasses this rule!, see above..). If the greeting does not follow the spec, it will be rejected.
- Unauthorized pipelining If the sending computer sends commands before we've given them permission to, reject them. Reputable software doesn't do that.
- Greylisting or Tempfailing - the act of giving people who are new to you a temporary failure. Chances are, it is a drive-by spamming. In this case, we will reject *new* contacts for a period of time. http://projects.puremagic.com/greylisting/ describes greylisting; we do use a variation of what is described (with diferent delays).
- Recipient's address, is it our customer? We accept almost all mail that is to our customers, unless it looks to be patently and obviously bogus. If our heurestics are too aggressive, and your mail is being blocked to a gigo.com customer, send email to postmaster@gigo.com, noting the sender address, and approximate time tha the mail was sent. We will gladly make exceptions for anyone who can even read the bounce message since you are obviously genuine.
- Sender's internet address (not email address) - is it a dialup modem? If you are a customer of a large dialup server, we will recognize that you are a modem user. If you are a modem user, you should be sending your mail via your ISP's mail server. If you are trying to send it to us directly, you are 99.99% likely to be a spammer - and we block it until you send mail via your ISP's mail server instead. This is notably true for Earthlink, UUNET, etc customers. We are willing to make exceptions.
- Sender's internet address and email address, are they an appropriate match? An example of this is AOL mail. We will gladly take AOL mail. That isn't a problem. However... we do verify that it is coming from an AOL server. If it is not from an AOL server, we will refuse the mail. AOL members are only supposed to send mail via the AOL service. We have similiar rules for several ISP's that are commonly used as false addresses in spam sent from all around the world. To reitterate: We don't mind mail from AOL, etc. We just enforce that it truely came from the right ISP's mail server.
We have several other tricks up our sleeves as well, which are in our best interest to not publish.. But ultimately, we check only the envelope, so the privacy of the message is preserved. Users are responsible for their own content filtering, if any. Users do have the option of having SpamAssassin prescreen their mail. If you are one of my users, and want to enable SA, then go here.
On request, we can and will put a filter on a user's account so that they will have html, RTF, and attachments stripped entirely. If you are one of the gigo.com users who store mail locally at gigo, and would like this, mail postmaster@gigo.com with your request.
your filter blocked us wrongly!
We'll be the first to say that our rules might accidentically refuse something that is legitimate. We do our best to avoid that. However, the dynamics of the internet do change. Sites that used to be blocked change ownership and purpose; internet addresses get owned by new people; new sites that appear to be mail relays are legimitate (blue mountain is an example).
If you had a message you sent get refused by gigo.com, and the message refered you to this page, we will be more than happy to make an exception for you. What we need to know is who you are, who you were sending to, and approximately when you sent it (down to the nearest day is fine). We'll verify the logs and figure out the best way to change our rules, whether it is a one-off case or a complete policy change.
If you are a gigo.com user, and you do not want any filtering done at all at this level, also send email. We can easily not filter mail for anyone who asks. If you want this to happen, we suggest a technique called whitelisting to keep your main inbox spam-free.
Whitelisting is the act of looking for the mail you *expect* to get and sending it to your main mailbox - and defering all other mail to a different mailbox so that it doesn't interrupt you with the same immediacy as your inbox. A sample script would be find-email-in-sentmail. This script is available for download but no tech support for it is available.
Requests should go to postmaster@gigo.com.
Requests to the postmaster are never filtered by gigo.com. However, it may
take up to a day to respond, depending on circumstances. gigo.com is a
hobby system providing free services to various individuals and hobby
groups; it provides no income. As such the postmaster has a paying job that
has to come first.
