Selfish is an attack on the integrity of the Bitcoin network. This is where one miner, or mining pool, does not publish and distribute a valid solution to the rest of the network.
The selfish miner then continues to mine the next block and so on maintaining its lead. When the rest of the network is about to catch up with the selfish miner, he, or they, then release here portion of solved blocks into the network.
The result is that their chain and proof of work is longer and more difficult so the rest of the network adopts their block solutions and they claim the block rewards.
The selfish mining attack is a method for mining pools to increase their returns by not playing fair. Although this can be seen in some crypto currencies where pool shares are not so fairly distributed it is more difficult to carry off with Bitcoin.
It is also advantageous for a mining pool to increase in size to be able to perform selfish mining. At over 51% of network power the returns earned by a selfish mining pool are justified as probabilistically they can maintain their advantage in terms of hashing the next block. As the mining pool controls the majority of the network hashing power it can also censor or nullify certain transactions essentially holding the network to ransom.
There have been BIPS, Bitcoin Improvement Proposals, to lower the probability of a selfish mining attack such as randomly assigning miners to various branches when a fork occurs – or alternatively providing a threshold limit to which a mining pool can reach. This is the same as Governments trying to halt natural monopolies to allow competition.
A further solution is to discriminate against a block depending on the timestamp it was released – so if a miner releases a long list of blocks in one shot – then the rest of the network would weight their validity against the timestamp they were hashed and the timestamp they were reported to the network.