Communities

Writing
Writing
Codidact Meta
Codidact Meta
The Great Outdoors
The Great Outdoors
Photography & Video
Photography & Video
Scientific Speculation
Scientific Speculation
Cooking
Cooking
Electrical Engineering
Electrical Engineering
Judaism
Judaism
Languages & Linguistics
Languages & Linguistics
Software Development
Software Development
Mathematics
Mathematics
Christianity
Christianity
Code Golf
Code Golf
Music
Music
Physics
Physics
Linux Systems
Linux Systems
Power Users
Power Users
Tabletop RPGs
Tabletop RPGs
Community Proposals
Community Proposals
tag:snake search within a tag
answers:0 unanswered questions
user:xxxx search by author id
score:0.5 posts with 0.5+ score
"snake oil" exact phrase
votes:4 posts with 4+ votes
created:<1w created < 1 week ago
post_type:xxxx type of post
Search help
Notifications
Mark all as read See all your notifications »
Q&A

Welcome to Software Development on Codidact!

Will you help us build our independent community of developers helping developers? We're small and trying to grow. We welcome questions about all aspects of software development, from design to code to QA and more. Got questions? Got answers? Got code you'd like someone to review? Please join us.

Comments on Uniform floating point from unsigned integer

Post

Uniform floating point from unsigned integer

+4
−2

How can I turn uniform samples of unsigned n-bit (n=16/32/64) integers (over the whole range of integers) into uniform samples of n-bit IEEE floats in [0, 1]? By uniform, I mean the continuous uniform distribution. All bounds are inclusive.

It needs to be accurate enough for stats/ML.

I'm using the StableHLO API, but an answer in e.g. C with equivalent maths and bit ops would be trivial to translate. Note this API can bitcast naively to float (i.e. reinterpret bits as they are as a float), and cast to float using the (approx) numerical value.

NB. I assume I can scale [0, 1] to [a, b] as samples * (a - b) + b since that's what XLA does (though I will have to be careful about overflow).

History

4 comment threads

Undeleted because of timing: an answer was being added as the question was being deleted. (1 comment)
Definition of "uniform" (12 comments)
Uniform over what ranges, exactly? (4 comments)
The most naive "bitcast" (7 comments)
Definition of "uniform"
trichoplax‭ wrote 8 months ago · edited 8 months ago

Could you give more detail of what your requirements are? Specifically, what does "uniform" mean in this context?

Over unsigned integers I might assume that this means uniformly randomly distributed samples (every representable integer is equally likely to appear in the sample), but for floating point numbers I don't have a single obvious definition. If every representable floating point number is equally likely, then each sampled number will be likely to be small, since there are more representable small numbers than large numbers (the gap between representable numbers is roughly proportional to the size of the numbers). If you want numbers that are evenly distributed as they would be with integers, then either some of the small numbers will need to not appear, or some of the large numbers will need to appear more frequently.

trichoplax‭ wrote 8 months ago

If you're concerned about integers, the upper end of the range of floats is exclusively (non-consecutive) integers. Are you looking to limit to the upper end of the unsigned integer range, leaving the larger floating point numbers unused?

deleted user wrote 8 months ago · edited 8 months ago

Uniform in the mathematical sense U[a, b], i.e. the expected number of samples is the same for any unit interval, or a flat line on a bar chart. I don't really understand your second comment, but I want to be able to get numbers at any size up to the max float.

For context, I expect this will be modified later to be rescaled to e.g. U[-5, 10], but I am assuming a simple multiplication and addition will suffice for this.

trichoplax‭ wrote 8 months ago

My second comment might be relevant if you're looking to scale to a different range. The gaps between floating point numbers are not consistent. For example, using 32 bit floating point:

  • The next float up from 1 is 1.0000001
  • The next float up from 10 is 10.000001
  • The next float up from 100 is 100.00001
  • The next float up from 1000 is 1000.00006
  • The next float up from 10000 is 10000.001
  • The next float up from 100000 is 100000.01
  • The next float up from 1000000 is 1000000.06
  • The next float up from 10000000 is 10000001 (a gap of 1)
  • The next float up from 100000000 is 100000010 (a gap of 10)

This means not only are all of the numbers above a certain point integers, but the gaps keep getting bigger for bigger numbers. There are no floats (in 32 bit) between 100000000 and 100000010.

trichoplax‭ wrote 8 months ago

If you just want a uniformly distributed sample in [-5, 10] you might be better off calculating this directly rather than trying to fit the whole unsigned integer range into that smaller range. If there's a reason you need it to start from the unsigned integers it might be useful to know that background information, so we know what aspects you need to preserve.

deleted user wrote 8 months ago · edited 8 months ago

I'm wrapping StableHLO, using rng_bit_generator. I assume it would be simpler to use that than build a sampler from scratch.

trichoplax‭ wrote 8 months ago

That specification also lists rng. Although it's deprecated, it has a UNIFORM option, which sounds like what you want. Would looking at its implementation help you write something that is not at risk of removal?

deleted user wrote 8 months ago

I like that idea

deleted user wrote 7 months ago

I've narrowed the question to U[0, 1]

trichoplax‭ wrote 7 months ago

It might help to specify how close to uniform you need this to be, as answers will vary significantly based on this information.

If you only need a roughly uniform distribution, then once you have an implementation that works for the range [0, 1], you can scale to other small ranges, such as multiplying by 5 and subtracting 2 to give the range [-2, 3]. This will no longer be uniform, but may be close enough for many cases (such as simple games).

If you need strong assurances of uniformity then you'd instead need a function that takes the bounds as inputs, because scaling the output of a function that outputs in the range [0, 1] will lose these assurances. The various problems that this would introduce may not matter depending on your intended usage, which is why it is worth specifying.

trichoplax‭ wrote 7 months ago · edited 7 months ago

As a simple example of why this would no longer be uniform, consider scaling from uniform output on [0, 1] by multiplying by 2. The outputs will all be in the range [0, 2], but there will be exactly the same number of possible outputs as there were in the range [0, 1]. This means that only half of the floats in the range [0, 2] will be possible to output. By multiplying by 2, the numbers in [0, 1] will have been spaced twice as far apart in [0, 2], without filling in the gaps. For many cases this won't matter, but it is important to be aware of this fact when deciding if this is sufficient for your purpose.

deleted user wrote 7 months ago · edited 7 months ago

ok, updated to U[a, b], since this is for stats/ML