What is relationship between machine learning and data mining?

This is not an easy question because there is no common agreement on what “Data Mining” means. But, I am going to say that I disagree with the answer from Wikipedia that Yuvraj Singla points to. I don’t think saying that machine learning focuses on prediction is accurate at all although I mostly agree with the definition of Data Mining focusing on the discovery of properties on the data. So, let’s start with that: Data Mining is a cross-disciplinary field that focuses on discovering properties of data sets. (Forget about it being the analysis step of “knowledge discovery in databases” KDD, this was maybe true years ago, it is not anymore).

On the other hand Machine Learning is a sub-field of data science that focuses on designing algorithms that can learn from and make predictions on the data. Machine learning includes Supervised Learning and Unsupervised Learning methods. Unsupervised methods actually start off from unlabeled data sets, so, in a way, they are directly related to finding out unknown properties in them (e.g. clusters or rules). It is clear then that machine learning can be used for data mining. However, data mining can use other techniques besides or on top of machine learning. Btw, to make things even more complicated, now we have a new term, Data Science, that is competing for attention, especially with Data Mining and KDD. Even the SIGKDD group at ACM is slowly moving towards using Data Science. In their website, they now describe themselves as “The community for data mining, data science and analytics”. My bet is that KDD will disappear as a term pretty soon and data mining will simply merge into data science.

Data mining isn’t a new invention that came with the digital age. The concept has been around for over a century, but came into greater public focus in the 1930s. According to Hacker Bits, one of the first modern moments of data mining occurred in 1936, when Alan Turing introduced the idea of a universal machine that could perform computations similar to those of modern-day computers.

Forbes also reported on Turing’s development of the “Turing Test” in 1950 to determine if a computer has real intelligence or not. To pass his test, a computer needed to fool a human into believing it was also human. Just two years later, Arthur Samuel created The Samuel Checkers-playing Program that appears to be the world’s first self-learning program. It miraculously learned as it played and got better at winning by studying the best moves. We’ve come a long way since then. Businesses are now harnessing data mining and machine learning to improve everything from their sales processes to interpreting financials for investment purposes. As a result, data scientists have become vital employees at organizations all over the world as companies seek to achieve bigger goals with data science than ever before.

With big data becoming so prevalent in the business world, a lot of data terms tend to be thrown around, with many not quite understanding what they mean. What is data mining? Is there a difference between machine learning vs. data science? How do they connect to each other? Isn’t machine learning just artificial intelligence? All of these are good questions, and discovering their answers can provide a deeper, more rewarding understanding of data science and analytics and how they can benefit a company.

Both data mining and machine learning are rooted in data science and generally fall under that umbrella. They often intersect or are confused with each other, but there are a few key distinctions between the two. Here’s a look at some data mining and machine learning differences between data mining and machine learning and how they can be used. One key difference between machine learning and data mining is how they are used and applied in our everyday lives. For example, data mining is often used by machine learning to see the connections between relationships. Uber uses machine learning to calculate ETAs for rides or meal delivery times for UberEATS.

Datamining can be used for a variety of purposes, including financial research. Investors might use data mining and web scraping to look at a start-up’s financials and help determine if they want to offer funding. A company may also use data mining to help collect data on sales trends to better inform everything from marketing to inventory needs, as well as to secure new leads. Data mining can be used to comb through social media profiles, websites, and digital assets to compile information on a company’s ideal leads to start an outreach campaign. Using data mining can lead to 10,000 leads in 10 minutes. With this much information, a data scientist can even predict future trends that will help a company prepare well for what customers may want in the months and years to come.

Machine learning embodies the principles of data mining, but can also make automatic correlations and learn from them to apply to new algorithms. It’s the technology behind self-driving cars that can quickly adjust to new conditions while driving. Machine learning also provides instant recommendations when a buyer purchases a product from Amazon. These algorithms and analytics are constantly meant to be improving, so the result will only get more accurate over time. Machine learning isn’t artificial intelligence, but the ability to learn and improve is still an impressive feat.

Machine learning, on the other hand, can actually learn from the existing data and provide the foundation necessary for a machine to teach itself. Zebra Medical Vision developed a machine learning algorithm to predict cardiovascular conditions and events that lead to the death of over 500,000 Americans each year. Machine learning can look at patterns and learn from them to adapt behavior for future incidents, while data mining is typically used as an information source for machine learning to pull from. Although data scientists can set up data mining to automatically look for specific types of data and parameters, it doesn’t learn and apply knowledge on its own without human interaction. Data mining also can’t automatically see the relationship between existing pieces of data with the same depth that machine learning can.

How is bitcoin mining done?

When you hear about bitcoin "mining," you envisage coins being dug out of the ground. But bitcoin isn't physical, so why do we call it mining? Because it's similar to gold mining in that the bitcoins exist in the protocol's design (just as the gold exists underground), but they haven't been brought out into the light yet (just as the gold hasn't yet been dug up). The bitcoin protocol stipulates that 21 million bitcoins will exist at some point. What "miners" do is bring them out into the light, a few at a time.

They get to do this as a reward for creating blocks of validated transactions and including them in the block chain. Bitcoin miners help keep the Bitcoin network secure by approving transactions. Mining is an important and integral part of Bitcoin that ensures fairness while keeping the Bitcoin network stable, safe and secure. Backtracking a bit, let's talk about "nodes." A node is a powerful computer that runs the bitcoin software and helps to keep bitcoin running by participating in the relay of information. Anyone can run a node, you just download the bitcoin software (free) and leave a certain port open (the drawback is that it consumes energy and storage space – the network at time of writing takes up about 145GB). Nodes spread bitcoin transactions around the network. One node will send information to a few nodes that it knows, who will relay the information to nodes that they know, etc. That way it ends up getting around the whole network pretty quickly. Some nodes are mining nodes (usually referred to as "miners"). These group outstanding transactions into blocks and add them to the blockchain. How do they do this? By solving a complex mathematical puzzle that is part of the bitcoin program, and including the answer in the block. The puzzle that needs solving is to find a number that, when combined with the data in the block and passed through a hash function, produces a result that is within a certain range. This is much harder than it sounds.

How hard are the puzzles involved in mining? Well, that depends on how much effort is being put into mining across the network. The difficulty of the mining can be adjusted, and is adjusted by the protocol every 2016 blocks, or roughly every 2 weeks. The difficulty adjusts itself with the aim of keeping the rate of block discovery constant. Thus if more computational power is employed in mining, then the difficulty will adjust upwards to make mining harder.  And if computational power is taken off of the network, the opposite happens. The difficulty adjusts downward to make mining easier.

In addition to being the means of generating new bitcoin, bitcoin mining creates the blockchain that verifies bitcoin transactions. The block reward is gleaned by placing a new block on the block chain, which acts as an advancing public ledger of verified transaction. This is an essential function for bitcoin's operation as it enables the currency to be safely and predictably created without the centralized regulation in the form of a bank or federal government. Blocks must to be a validated by a proof-of-work (Bitcoin uses Hashcash), which can only be obtained by expending a great deal of processing power. Once a block is obtained a message is broadcast to the mining network and verified by all recipients.  In the earliest days of Bitcoin, mining was done with CPUs from normal desktop computers.  Graphics cards, or graphics processing units (GPUs), are more effective at mining than CPUs and as Bitcoin gained popularity, GPUs became dominant.  Eventually, hardware known as an ASIC, which stands for Application-Specific Integrated Circuit, was designed specifically for mining bitcoin.  The first ones were released in 2013 and have been improved upon since, with more efficient designs coming to market.  Mining is competitive and today can only be done profitably with the latest ASICs.  When using CPUs, GPUs, or even the older ASICs, the cost of energy consumption is greater than the revenue generated.

The difficulty of the calculation (the required number of zeroes at the beginning of the hash string) is adjusted frequently, so that it takes on average about 10 minutes to process a block. That is the amount of time that the bitcoin developers think is necessary for a steady and diminishing flow of new coins until the maximum number of 21 million is reached (expected sometime in 2140).

If you've made it this far, then congratulations! There is still so much more to explain about the system, but at least now you have an idea of the broad outline of the genius of the programming and the concept. For the first time we have a system that allows for convenient digital transfers in a decentralized, trust-free and tamper-proof way. The repercussions could be huge. What is the point of Bitcoin mining? This is something we're asked every day.

Traditional currencies--like the dollar or euro--are issued by central banks. The central bank can issue new units of money at any time based on what they think will improve the economy. With Bitcoin, miners are rewarded new bitcoins every 10 minutes. The issuance rate is set in the code, so miners cannot cheat the system or create bitcoins out of thin air. They have to use their computing power to generate the new bitcoins. Miners include transactions sent on the Bitcoin network in their block. A transaction can only be considered secure and complete once it is included in a block. Because only a when a transaction has been included in a block is it officially embedded into Bitcoin'sblockchain.

·         More confirmations are better for larger payments. Here is a visual so you have a better idea:

·         Payments with 0 confirmations can still be reversed! Wait for at least one.

·         One confirmation is enough for small Bitcoin payments less than $1,000.

·         Enough for payments $1,000 - $10,000. Most exchanges require 3 confirmations for deposits.

·         Enough for large payments from $10,000 - $1,000,000. Six is standard for most transactions to be considered secure.