An increasingly popular approach for analyzing neural data is to treat it as being constrained to and sampled from a manifold, which can then be characterized by its topology. The persistent homology method identifies the type and number of holes in the manifold thereby yielding functional information about the coding and dynamic properties of the underlying neural network. In this work we give examples of highly non-linear manifolds for which the persistent homology algorithm fails when it uses the Euclidean distance, because this distance is not a good approximation of the true distance distribution of the point cloud. We propose to use the geodesic distance because it yields a better approximation of the true distance distribution and can be used to identify highly non-linear features with persistent homology. To evaluate our method we model a circular manifold and show how the chosen metric determines the performance of the persistent homology algorithm. Furthermore we discuss the robustness of our method under different manifold properties and point out strategies for interpreting its results as well as some possible pitfalls of its application. We apply this analysis to neural data coming from the Visual Coding - Neuropixels dataset provided by the Allen Institute and find different manifolds with a non-trivial topology across regions. Finally, we discuss what these manifolds say about visual computation and how they depend on the structure of stimulus space.