My point is that we're not just talking about A/B of 100, but rather A/B of 10000000000000 or more. If you're inverting the ratio we're estimating, then instead of estimating the value at 0, we're estimating it at infinity (or, okay, if you want to use Laplace smoothing, then a million, which is far too low).
There are situations where you would use rejection sampling because generating 100x more random numbers is much cheaper than doing the complex calculations required to accurately model the space in question. Maybe 50 dimensions (and the 13 orders of magnitude difference) isn't big enough to raise those concerns. If we instead talk about 100 dimensions, then we're dealing with a difference of 40 orders of magnitude, and if you tell me that still doesn't matter, then I don't know what to say.
You haven't addressed my question at all. If you think estimating 1e-1000 is a problem, then estimating 1e+1000 shouldn't be. But one is just the inverse of the other. What's the problem with this argument, you haven't answered it at all.
> If you think estimating 1e-1000 is a problem, then estimating 1e+1000 shouldn't be.
They're both problems. If you want to estimate 1e1000 via sampling individual points, then you need at least that order of magnitude of samples. If all of your data points fall in one class, then it doesn't matter what you're trying to calculate from that.
As I said: "If you're inverting the ratio we're estimating, then instead of estimating the value at 0, we're estimating it at infinity (or, okay, if you want to use Laplace smoothing, then a million, which is far too low)."
> If you want to estimate 1e1000 via sampling individual points, then you need at least that order of magnitude of samples.
Ok so if the ratio is 1/2 how many samples do you need?
I mean, yes, you can estimate this for low dimension. It's a bad idea given how slow the convergence is, but you can do it.
My entire point is that this becomes infeasible very quickly for numbers that are not all that big.
First off, you didn't answer my question which to restate is: if the ratio is 1/2, how many samples do you need.
Second, your claim that it depends on dimension is wrong, given the ratio of sets, dimension doesn't matter. If the ratio is 1/2 then you'll reject one half of the times regardless of dimension, and so by your own argument it's "very efficient"
> First off, you didn't answer my question which to restate is: if the ratio is 1/2, how many samples do you need.
Are we just trying to estimate the ratio? To what desired accuracy? If you have 100 samples and find that 50 points landed in your smaller set, okay, you can be 95% sure that the ratio is between 40% and 60%. That's not a very good estimate. If you want to drive those error bars down by 2x, you need 4x as many samples.
> Second, your claim that it depends on dimension is wrong, given the ratio of sets, dimension doesn't matter. If the ratio is 1/2 then you'll reject one half of the times regardless of dimension, and so by your own argument it's "very efficient"
When your sets in question are the volume enclosed by the n-dimensional unit hypersphere and the volume enclosed by the corresponding hypercube, which is what I assume we've been discussing this whole time, you do not get to pick what your ratio is between your sets for a given choice of n. If you're dealing with a 3-dimensional unit sphere and picking three random values uniformly between 0 and 1 (i.e. constrained to the positive octant), you will land within the hypersphere with probability pi/6 (close to your one-half ratio). You can't decide "okay now my ratio is 99%" unless you change how you draw your samples.
You can draw however many samples you want from the 100-dimensional unit hypercube to estimate the volume of the 100-dimensional unit hypersphere, and all you'll ever get is "the ratio is somewhere between 0 and some very small number, with X% probability".
Either way, as I have said multiple times, you are completely missing the point of rejection sampling by overindexing on the toy example that I explicitly stated is not a good use of rejection sampling.
> Are we just trying to estimate the ratio? To what desired accuracy?
Exactly. Number of samples depends on volume ratio and accuracy. It depends on dimension only through these two numbers. You got there in the end, proud of you.
You continue to miss the point. In this specific context, volume ratio scales super-exponentially with dimension, and unless you're willing to have accuracy also drop super-exponentially (as in, go very very quickly to zero), then you are wasting your time by trying to perform rejection sampling in a context where you are statistically never going to hit the target set.
No, I didn't miss that point. I'm well familiar with hyperspheres. And yet, the thing you described is only incidentally related to dimension.
Do you know how to program? It’s super easy to write a very simple rejection-based program to estimate the volume of a hypersphere (you can do it in <10 lines with numpy). Try it yourself for dimensionality 50 and see how long it takes before the estimate rises above precisely 0
Read my conversation with the other person. If you sample N times you'll be able to put a bound on the ratio, and that is the same regardless of dimension. To your example: if the ratio between sets in D=50 is 1e-50 and you ask how long it takes that your estimate bound doesn't contain zero, that will take a long time. Now if I ask you to estimate the ratio between the 2D circle and the 2D square with 50 decimal place, it will take the same time. Therefore, dimension doesn't enter here. This is a general property of Monte Carlo.