Connecticut has two large state parks on long island sound, Hammonasset Beach and Rocky Neck State Park. http://dep.state.ct.us/stateparks/camping/camping.htm
I have never fished Rocky Neck but I know that Hammonasset has a jetty that is well used by fishreman. I have caught flounder and have seen others catch blufish and striped bass there. There is a bait shop very close to Hammonasset. I have not been fishing there yet this year so I could not tell you how the fishing is.