Ketan's Home

January 31, 2016

16 for ’16: What you must know about Hadoop and Spark right now

Filed under: Uncategorized — Tags: , — ketan @ 7:12 PM

The biggest thing you need to know about Hadoop is that it isn’t Hadoop anymore.

Between Cloudera sometimes swapping out HDFS for Kudu while declaring Spark the center of its universe (thus replacing MapReduce everywhere it is found) and Hortonworks joining the Spark party, the only item you can be sure of in a “Hadoop” cluster is YARN. Oh, but Databricks, aka the Spark people, prefer Mesos over YARN — and by the way, Spark doesn’t require HDFS.

Yet distributed filesystems are still useful. Business intelligence is a great use case for Cloudera’s Impala and Kudu, a distributed columnar store, is optimized for it. Spark is great for many tasks, but sometimes you need an MPP (massively parallel processing) solution like Impala to do the trick — and Hive remains a useful file-to-table management system. Even when you’re not using Hadoop because you’re focused on in-memory, real-time analytics with Spark, you still may end up using pieces of Hadoop here and there.

To read this article in full or to leave a comment, please click here

from JavaWorld

January 28, 2016

Stop Driving Laser Cutters with 3D Printer Software!

Filed under: Uncategorized — Tags: — ketan @ 12:55 AM

Laser cutters are fantastic pieces of equipment, and thanks to open-source improvements in recent years, are getting even cheaper to make. It can be as simple as throwing a high-powered laser diode onto the head of your 3D printer! With so many home-brew designs out there, wouldn’t it be nice if there was some all-encompassing open-source, laser-cutter controller software? Well, as it turns out — there is, and it’s called LaserWeb.

What started as a simple personal project by [Peter van der Walt] has recently grown into a very formidable piece of software with over 10 contributors in just three …read more

from Hackaday

Cheap WiFi Devices are Hardware Hacker Gold

Filed under: Uncategorized — Tags: , — ketan @ 12:55 AM

Cheap consumer WiFi devices are great for at least three reasons. First, they almost all run an embedded Linux distribution. Second, they’re cheap. If you’re going to break a couple devices in the process of breaking into the things, it’s nice to be able to do so without financial fears. And third, they’re often produced on such low margins that security is an expense that the manufacturers just can’t stomach — meaning they’re often trivially easy to get into.

Case in point: [q3k] sent in this hack of a tiny WiFi-enabled SD card reader device that he and his compatriots …read more

from Hackaday

January 27, 2016

Using Arduino For Quadcopter Spectrum Analyzers

Filed under: Uncategorized — Tags: — ketan @ 3:55 PM

First-person-view (FPV) flying, by adding a camera, video transmitter, and video goggles to the meat on the ground, is one of the best ways to experience remote-controlled flight. For just a few hundred dollars, it’s the closest thing you’re going to get to growing wings and flying through the trees of your local park. One of the most popular and cheapest ways to go about this is the Boscam RX5808 wireless receiver – a $9 module able to pull down video from an aircraft over 5.8GHz radio. Stock, this radio module is just okay, but with a few modifications, it …read more

from Hackaday

January 25, 2016

Turning Your CNC Machine into a Pen Plotter

Filed under: Uncategorized — Tags: , — ketan @ 1:24 AM

plotterPen_4A simple wood and plastic adapter that turns a ShopBot into a pen plotter

Read more on MAKE

The post Turning Your CNC Machine into a Pen Plotter appeared first on Make: DIY Projects, How-Tos, Electronics, Crafts and Ideas for Makers.

from Make: DIY Projects, How-Tos, Electronics, Crafts and Ideas for Makers

January 23, 2016

Let’s Play…Wheel of Solder!

Filed under: Uncategorized — Tags: , — ketan @ 11:59 AM

Solder is solder right? Just spin the wheel and whatever comes up will work fine. Well, not so fast. If you’re new to electronics, or are looking at getting started, there is a bit to learn first. [Mr Carlson] has the info you need with this youtube video you can watch after the break.

He begins with a discussion of solder diameter. For most through hole work, something around 0.03 inch is pretty universal. When your ready to step up to SMD work, we find 0.02″ inch to be a much better match to the smaller pad sizes. [Mr Carlson] …read more

from Hackaday

January 18, 2016

Custom Siri Automation with HomeKit and ESP8266

Filed under: Uncategorized — Tags: — ketan @ 11:52 AM

Knowing where to start when adding a device to your home automation is always a tough thing. Most likely, you are already working on the device end of things (whatever you’re trying to automate) so it would be nice if the user end is already figured out. This is one such case. [Aditya Tannu] is using Siri to control ESP8266 connected devices by leveraging the functionality of Apple’s HomeKit protocols.

HomeKit is a framework from Apple that uses Siri as the voice activation on the user end of the system. Just like Amazon’s voice-control automation, this is ripe for exploration. …read more

from Hackaday

January 15, 2016

Inject Packets with an ESP8266

Filed under: Uncategorized — Tags: , — ketan @ 9:10 PM

[Kripthor] sent us a link to his blog where he writes the Hello World of low-level networking. Basically he’s constructing his own packet and sending it. By itself this isn’t a bad thing. You could use this power for all sorts of networks-diagnostic good. And so, despite the ominous name of his blog post “ESP8266 Jamming”, he’s not really doing anything that bad — he’s just creating many fake WiFi beacon frames and sending them out every so often.

Which can apparently do bad things to some vulnerable routers. Who knew? Want to test yours?

Naturally we wanted to see …read more

from Hackaday

January 14, 2016

How To Activate A Clean Install Of Windows 10 With A Windows 7 or 8 License Key

Filed under: Uncategorized — Tags: , — ketan @ 8:26 PM

Windows 10 is free for Windows 7/8 users and the upgrade process is incredibly smooth. You need only download the right files and chances are they’re already there. Create the media tool of your choice and you’re good to go. The Windows 10 upgrade process will let you keep your apps and files intact and […]

Read How To Activate A Clean Install Of Windows 10 With A Windows 7 or 8 License Key by Fatima Wahab on AddictiveTips – Tech tips to make you smarter

from AddictiveTips

Resolve OutOfMemoryError With Excel Export in the Apache POI Stream API

Filed under: Uncategorized — Tags: — ketan @ 7:35 AM

Whenever we try to export a huge excel spreadsheet (for example: around 200,000-300,000 records), most of the time we end up with an OutOfMemoryError:JavaHeapSpace. We also consume more time and processing power to export that much of data. The main reason for this kind of problem is that the prior version of Apache POI (prior to 3.8) does not provide a proper solution for this kind of situation, and I also have other issues with the API design in those versions. I’ve sometimes faced situations where we couldn’t support more than 65000 rows of data during excel exports with prior versions of POI. But with version 3.8 and higher, there are solutions for all these problems.

To resolve the memory issues and performance issues of Excel exports, they have utilized a stream API to design to support large data exports. With the stream API we can flush only few rows of data into memory and the reamining rows can be flushed to the hard memory (permanent memory). In this example, you can easily see how it supports larger datasets. I wrote this utility for supporting almost 200,000 records with one of my applications. I hope it will help many who are in search of this kind of solution. I built this solution with Spring MVC.

from DZone Java Zone

Older Posts »

Create a free website or blog at