On-Demand HTTP Live Streaming Made Easy

As a developer, sometimes I want to share a video inside of an existing web application. In the past, I have relied on progressive download streaming, for simplicity. Unfortunately, there are some limitations with progressive streaming. Generally, one has to limit quality to get a video that will play on most devices. Seeking can also be an issue on longer videos.

Back in 2009 Apple released a solution to this problem called HTTP Live Streaming. While the name indicates it’s only for live events, it’s quite adaptable to recorded content as well. Unfortunately, the software to prepare these streams is usually expensive, cumbersome, or doesn’t provide good quality. Most tutorials will point you at a cloud service like YouTube or Vimeo. Some will recommend using a product like Sorensen Desktop. Others may recommend using something like Zencoder. That’s all a bit excessive, as this can be a simple process you can do on your computer! Once the stream is prepared, you don’t need anything fancy to publish it. Any web host will do, be it Amazon S3, Dreamhost or your own private server.

If you’ve got a Windows machine, grab this zip file. I’ll wait. Unzip it somewhere with some free space. Create a folder named “input” and drop a video file or two in there, and run the transcode.bat file. Mac & Linux users can try the bash version and let me know if it’s working.

In a few minutes, depending on the speed of your computer, you should have a new folder with the name of your video file. You can now upload that folder up to your website and you’re ready to go! Check out the example below or visit it directly here.

AlexOn-Demand HTTP Live Streaming Made Easy
read more

Updated WhatIsMyIP for Bash

A long time ago, I posted a snippet to get your IP address using Bash. Turns out the service I was using in that snippet has gone away, like so many others. Now to give you a few alternatives!

If you’ve got Python handy (you probably do), you can use my ip.coda.to service’s API to output way more information that just an IP address. curl ip.coda.to/api | python -m json.tool will give you quite a chunk of information! If you want to add that to your bash profile, use the snippet below.

ipinfo ()
{
  url="http://ip.coda.to/api"
  if [ "$1" ]; then
    url+="/$1"
  fi
  curl --location "$url" | python -m json.tool
}

If you just want an IP, there are a few good ways to get it. OpenDNS has a special DNS zone if you’re using their resolvers, so as long as you have a working nslookup (Windows, macOS, and most unixes with the bind-utils package).

nslookup myip.opendns.com 208.67.222.222

If you want to use curl, there’s a few options.

curl api.ipify.org
curl ifconfig.me/ip
curl wtfismyip.com/text
curl ipinfo.io/ip
curl whatismyip.akamai.com

 

AlexUpdated WhatIsMyIP for Bash
read more

Mandrill Alternative Roundup

Recently, a popular transactional email provider decided to alter the terms of their agreement with many of their customers. This post is specifically for people targeted by that announcement, and is to provide a quick summary of some alternative services that might be of interest to you.

Postmark

Postmark has been doing this whole thing since 2010, a full 2 years before Mandril was in Beta. I find them to be the ideal solution if either deliverability to the inbox or pricing flexibility is paramount. For low-volume use, they’re a bit on the expensive side but they’re great at what they do.

SparkPost

SparkPost is by the same folks who build the Port25 MTA used by the biggest e-mail companies out there (including Salesforce and MailChimp). That means they’re the experts in using that software and SparkPost serves as an environment for them to tune, tweak and do other production-y R&D to improve their core product. That means you’re getting access to some of the foremost experts on email delivery (the folks MailChimp calls for help), and in my quick testing of the platform some of the best delivery speed available.

Pricing is a little funky, but for low-volume senders this is going to be my go-to recommendation because up to 100,000 e-mails /month is completely free. Unfortunately, there is no overage option on that plan, and if you need to send a single e-mail over that number you have to go with a paid plan starting at $44.99 /month for 200,000 e-mails and $0.20 per thousand overage. This is still significantly cheaper than Postmark, but SparkPost is a lot newer service and the free offer is subject to change at any time.

SendGrid

SendGrid has been in the e-mail business a good while as well. They offer a starter tier of 12,000 e-mails /month for free, and 100,000 e-mails is just $19.95 /month. Bumping up to additional features and volumes start to include fairly hefty price premiums, but if the numbers work out they’ve been a great partner to many companies out there.

 

UPDATE: 3/9/16 – SparkPost has updated their pricing, now 100,000 e-mails are free or for $24.99 you can get the 100,000 e-mails with overages allowed at $0.20 per thousand. This is an excellent change!

UPDATE: Sometime in October – SparkPost has added the ability to add Dedicated IPs and overages to non-subscription plans!

AlexMandrill Alternative Roundup
read more

40 TB for $4k

I was working with one of my clients a few months ago making our project plans for the year when we hit the topic of storage. We were looking to add approximately 20TB of file storage and needed a place to back it up, but we wanted to back up to different technology than we were using for the raw file storage and we wanted to keep the budget as low as possible. I did some digging, and put together numbers for a system I had been planning in the back of my mind for a while and said “What if I could get you 40 TB of backup space in 1U for under $4,000?”

This particular client has been burned pretty badly more than once by large storage systems, so his initial response was “I’d say that sounds like a disaster waiting to happen.” I, of course, replied “maybe, but what are the chances of that disaster happening the same time we loose the primary storage arrays?”

Where do I sign up?

The key component of the build is this odball little barebones kit by ASRock’s server team (ASRock Rack) which squeezes 12 3.5″ SATA drives in a 1U rackmount enclosure. Just add RAM, HDDs and an OS and you’re running. My build specified Nas4Free as the OS, so all I needed for it was a USB drive, but we wanted a large amount of RAM for the best stability and performance so 32 GB of Crucial ECC ram was installed (2 16GB kits). For storage drives, I selected 4 TB HGST Deskstar NAS drives for their high reliability, performance and value. I ordered 2 spares to keep on-site, as HGST does not currently do advanced RMAs and given the number of drives in this system, failure is inevitable. The 12 drives are configured in a ZFS RaidZ2 for just over 40 TB of usable storage.

The final parts total ended up as:

1 x Barebone Kit @ 1,059.99
1 x USB Boot Drive @ 8.99
2 x RAM Kit @ 183.85
14 x 4TB HGST Drive @ 164.99

That brings the final parts total to $3,746.54 + Tax.

Caveats

This build didn’t go entirely without a hitch. The original RAM we ordered would not POST in the system, so it had to be sent back and replaced with Crucial. The RAM slots are kinda junky, so getting the memory seated took several tries. For some reason, whenever both LAN ports were connected to the network there ended up being a broadcast loop which actually took the network down (at 5pm on a Friday, but still). We also had a drive fail in the first week, which is pretty much expected with that many.

Performance

I’m currently using this system as an FTP server for the existing QNAP’s RTRR feature which is set to run at the completion of the normal backup window. Each of the 2 QNAPs is CPU limited at pushing about 50 MB/sec over to the new backup NAS, and this system hangs out at about 20-40% CPU usage while handling that. Scrubs and Re-silvers happen internally around 500-600 MB/sec, so throwing in a 10GB or Quad Port Gig Ethernet card might be worthwhile if it’s got the PSU power to run one and your network can take advantage of it.

Alex40 TB for $4k
read more