Surely it boils down whether this garden included in your lease and you have exclusive access to it or if it is communal land owned by the freeholder that you have access to. If it is the former then I think permission should not be unreasonably witheld (does your lease state you need permission and did you seek it?) If it is the latter then you have encroached on land that is not yours. We have access to communal gardens in our block, our freeholder would take a very dim view of residents erecting sheds and other structures in the gardens. I would also not point to the poor behaviour of other lessees to justify your case. As a director of a freehold company I would take a dim view of that. The whole issue is whether you have acted within your lease and whether your freeholder is being reasinable. That is it.