Test set RULs

Here are the RULs for the test set.

FAQ: After the challenge is over...

Several people have asked about what will happen with the data set and scoring after the challenge is over.

Scores will be published for the final test set (FTS) on the web site using the pseudonym of the team.

The data (i.e., the exact RULs) for the test set and FTS will be made available for download, to use as a benchmark for future algorithms.

Finally, the nature and origin of the data set will be revealed.

FAQ: Hey! People might cheat!!

As a bajillion people noted, allowing people to check the scores of the test set allows people to cheat by unfairly tuning their algorithm to improve their score. Our hope in providing public scores was to inspire a little friendly competition end motivate people to get the most out of their algorithms...

To correct this flaw in our contest design, we are still going to use the "test set" to update the leaderboard once each day. Think of the current test set as a validation set. There will be a "final test set" released on 19 May 2008. Algorithms will be scored on the basis of performance on the final test set only. Scores will not be released for the final test set until after the competition has closed.

FAQ: Why is there so little information about the system? Why are your answers so vague?

Why is there so little information about the system? Why are your answers so vague?

This contest is about data driven, platform independent PHM. The system itself is intended to be a black box – you have no real information about it, or domain knowledge, other than the data. We recognize that whenever domain knowledge is applied to a problem, superior results are obtained. However, there are many situations where there is data available – frequently not directly PHM related (e.g., control system data) – that can be used for PHM where little or no domain knowledge is easily obtainable. Techniques for estimating the remaining useful life of any system for which data are available have numerous applications in industry.

FAQ: What is the scoring function to be used?

What is the scoring function to be used?

d = estimated_RUL-actual_RUL;

score(d<0) = exp(-d(d<0)/13)-1;
score(d>0) = exp(d(d>0)/10)-1;

score = sum(score)

FAQ: Can you tell us if we can expect more than one fault mode?

Can you tell us if we can expect more than one fault mode?

No, but the data can...

FAQ: Is each operational cycle equivalent in terms of time or the amount of damage induced by the fault?

Is each operational cycle equivalent in terms of time or the amount of damage induced by the fault?

Each operational cycle is effectively the same duration across all units. As far as damage accumulated per cycle, you’ll have to look at the data...

FAQ: Can you tell us any more about when the system failure occurs?

Can you tell us any more about when the system failure occurs (not the initial fault, but the end failure)? Does it occur at the end of the (training) data set or just at some point during the data set (and additional data is captured afterward)?

The remaining useful life of the last operational cycle of each unit in the training data is zero. At this time, one or more limits for safe operation have been reached, and the unit may not be used for another operational cycle. Note that it is not a “hard” failure – the unit does not explode or anything – although we admit that would be much cooler. Rather, it is more akin to reaching the wear bars on a set of automobile tires, or diving to the maximum safe pressure in a submarine.

FAQ: Do the unit numbers in the training data correspond to the unit numbers in the test data?

Do the unit numbers in the training data correspond to the unit numbers in the test data? For example, is Unit #12 in the training set the same physical unit as Unit #12 in the test set?

No. The unit numbers are just arbitrary labels – there is no relationship between unit numbers in the training and test data.

FAQ: What are the operational settings corresponding to the remaining operational cycles for each unit in the test data?

What are the operational settings corresponding to the remaining operational cycles for each unit in the test data?

That is unknown. You will have to assume something...

FAQ: Is each sensor the same in each operational cycle or unit?

Is each sensor the same in each operational cycle or unit?

Yes! So if sensed value one is resistance in ohms, it is resistance in ohms throughout all operational cycles and across all units.

FAQ: What are the 21 sensors?

What are the 21 sensors?

This is intended to be data driven, platform independent PHM, so by design, you have no domain knowledge.

Welcome to the PHM Challenge blog!

Welcome to the PHM Challenge blog!
We will post FAQs and results here as necessary...