After several months of virtual radio silence in the mainstream press about the IRS scandal, the over-the-top coverage afforded Chris Christie’s Bridgegate fiasco reminded conservatives of the way many in the media downplayed the outrageous accounts of the government’s bias against conservative political groups. But the IRS affair got back into the news in its own right today as the Wall Street Journal reported that the FBI is unlikely to prosecute anyone for the practice in which organizations affiliated with the Tea Party and other conservatives causes were specifically targeted for discriminatory treatment.

This is bound to fuel further complaints from Republicans who have been frustrated with the administration’s low-key response to a scandal based in a policy they think may have been inspired by the president’s personal biases against conservative groups as well as the liberal belief that Tea Party-affiliated organizations don’t deserve non-profit status. After initially adopting a defensive tone and agreeing that any discrimination was wrong, the party line from the White House has been that the problems were the result of the mistakes made by rogue low-level officials working in Cincinnati and that any talk about a scandal is mere GOP propaganda. That position will be bolstered by the FBI decision. But, as the Journal’s report notes, there is a big problem with the investigation that was conducted: apparently nobody in the FBI has contacted any of the groups that were the object of the agency’s special attentions. This is likely to only deepen the cynicism felt by many on the right toward an administration that doesn’t seem particularly fired up about holding the tax collectors accountable for their misdeeds.

The sensitivity of this case is the product of both the blatantly political nature of the BOLO (be on the lookout for) orders that were sent out about groups seeking non-profit status for their public education efforts and the immense power of the IRS. That the IRS seemed to be following the administration’s marching orders in treating conservative efforts to take advantage of the change in the law after the Supreme Court’s Citizens United decision raised serious suspicions. But the revelations last spring about the agency’s inquisitions directed at Tea Partiers and delays in making decisions prompted congressional investigations that were, at least in part, short-circuited by the lack of candid answers from IRS officials like Lois Lerner who went so far as to invoke the Fifth Amendment when questioned by Congress last May.

It should be acknowledged that prosecutions in a case involving interpretations of policy would not be simple. But there appeared to be grounds to think laws involving the misuse and improper disclosure of taxpayer information were violated by those involved. However, the FBI seems to have taken a “no harm, no foul” approach to the case in which the lack of a smoking gun about the agency “hunting” conservatives rather than merely applying discriminatory policies appears to be working to stifle any impulse to prosecute.

However, this merciful approach to IRS personnel and their political superiors seems at variance with that agency’s usual approach to taxpayers. It is well known that citizens who claim to have been confused by complicated tax regulations or that they acted on the advice of their lawyers are generally shown no mercy by implacable IRS agents who haul taxpayers into court for minor violations of confusing statutes.

Even worse, the fact that the FBI chose not even to interview groups that have claimed discrimination casts doubt on the seriousness of the investigation and whether the normally indictment-happy Justice Department lacked the will to pursue the case.

As I wrote in December, the administration is hoping to put a close to this controversy by altering the rules to make it difficult, if not impossible for groups that aim at promoting political change—be it from a right-wing or a left-wing point of view—to become non-profits. While technically neutral, this change will have a disproportionate impact on conservative groups since they are far more dependent on 501(c) status than their rivals on the left. This should concern all Americans no matter their politics since giving the IRS that much power to suppress the free speech of political activists poses a grave threat to democracy. The way the agency has been used to regulate political activity is perhaps the most serious scandal here, but combined with the failure of the government to ensure that those who ordered and carried out discriminatory policies are held accountable makes it even worse.