If the proposed Islamic Center near Ground Zero gets off the ground, it will need hard hats to build it.

But there is a call to construction workers not to lift a hammer.

Construction worker Andy Sullivan is trying to get members of his trade to boycott building the project. He has started a website called “911hardhatpledge.com,” urging hardhats to sign up for a boycott.

“I feel it is hurtful, and it is disrespectful and that is exactly what this mosque will be if it is built,” says Sullivan. “It is totally a grass-roots effort,” he says, adding that “we started the postcard push where you can actually go to the site and send a postcard to a politician who is supporting the mosque and tell them, ‘Hey, we don’t think it is right.”

There was a mixed reaction from construction workers near Ground Zero about whether they would want to be part of the construction crew. More of them with whom we talked said they would not want to build it, than those who would be willing to be part of the construction. Some said they would have no choice if their employer got the contract, others said they simply would refuse.

The president of the Building and Construction Trades Council of Greater New York, Gary LaBarbera, told Fox News that the Council has not taken a formal position on the center.

The project’s sponsor, The Cordoba Initiative, did not return a request for comment.

Meanwhile, a gathering of Muslim groups from around the country met in New York City to find a way to deal with the controversy.

The Islamic Leadership Council of Greater New York supports the building of the center and is urging people to do likewise.

The groups have announced a “National Week of Dialogue” for next month, which they say will “combat bigotry and promote understanding and religious tolerance in America.”

The groups also noted that Americans of all faiths have the right to build houses of worship where they want.

The developer, Park51, issued statement that said, in part, that they are grateful for the support. “Our community remains committed to building bridges of understanding to our neighborhoods, to our city, and to the rest of America.”