> On Wednesday, March 29, 2023 at 10:05:22 AM UTC-7, Don Y wrote:
>> On 3/29/2023 6:23 AM, legg wrote:
>>> On Tue, 28 Mar 2023 07:14:16 -0700, Don Y
>>> <blocked...@foo.invalid> wrote:
>>>
>>>> On 3/28/2023 6:34 AM, legg wrote:
>>>>> On Mon, 27 Mar 2023 15:11:07 -0700, Don Y
>>>>> <blocked...@foo.invalid> wrote:
>>>>>
>>>>>> On 3/26/2023 8:51 PM, Don Y wrote:
>>>>>>> I'm planning on operating a couple of 1U servers "on their sides"
>>>>>>> (i.e., tall, deep and VERY skinny).
>
> Better to be up front. I.e. Front panel pointing up. At least for the Dell Poweredge. The fans are pushing air from back to front. They are very noisy, so i put in a resistor to slow them down.
If that's the case, it's the exact opposite of the airflow in most servers;
the front panel is in the "cold aisle" (where humans access the device)
and the back of the server is the warm side (exhaust).
That is definitely the case in all of my servers (these are PE R610's).
>>>>>>> Of course, this is normal to their expected operating orientation.
>>>>>>>
>>>>>>> I figure the disks won't mind being operated in that orientation
>>>>>>> as I have shelfs with (the same) drives oriented that way.
>
> If you can afford SSD, they would be better.
Can't afford ~200TB of SSD (2x4x15 drives being managed by these servers).
Nor do I need that sort of access time *or* power savings as this is for
my cold archive.
> I also run laptops sideway, with fan pointing up. I can see at least 10°C cooler from internal sensors.
I'm not sure I'd trust a laptop in a weird orientation.
I've had (consumer) laptop drives fail when operated (continuously) in
"non-flat" orientations (though installed in things other than laptops).
Reply by Ed Lee●March 29, 20232023-03-29
On Wednesday, March 29, 2023 at 10:05:22 AM UTC-7, Don Y wrote:
> On 3/29/2023 6:23 AM, legg wrote:
> > On Tue, 28 Mar 2023 07:14:16 -0700, Don Y
> > <blocked...@foo.invalid> wrote:
> >
> >> On 3/28/2023 6:34 AM, legg wrote:
> >>> On Mon, 27 Mar 2023 15:11:07 -0700, Don Y
> >>> <blocked...@foo.invalid> wrote:
> >>>
> >>>> On 3/26/2023 8:51 PM, Don Y wrote:
> >>>>> I'm planning on operating a couple of 1U servers "on their sides"
> >>>>> (i.e., tall, deep and VERY skinny).
Better to be up front. I.e. Front panel pointing up. At least for the Dell Poweredge. The fans are pushing air from back to front. They are very noisy, so i put in a resistor to slow them down.
> >>>>>
> >>>>> Of course, this is normal to their expected operating orientation.
> >>>>>
> >>>>> I figure the disks won't mind being operated in that orientation
> >>>>> as I have shelfs with (the same) drives oriented that way.
If you can afford SSD, they would be better.
I also run laptops sideway, with fan pointing up. I can see at least 10°C cooler from internal sensors.
Reply by Don Y●March 29, 20232023-03-29
On 3/29/2023 10:05 AM, Don Y wrote:
> The 30 inch front-to-back dimension is the pisser. I'm mounting
> these "on edge" as "bookends" under my benches (where I can
> tolerate the extra "depth". The 24 inch devices are much easier
> to accommodate in their normal orientation (32 inch deep benches)!
> But, they tend to be less capable devices.
I have two stacks of 4 15-drive JBOD shelfs. Siting a server near
each will let me just talk to them (via the server) as a SAN. Or,
from dumber clients, as a glorified (customizable) NAS.
[I'm not fond of COTS NAS/SAN offerings as someone *else* decides
how those will work!]
Reply by Don Y●March 29, 20232023-03-29
On 3/29/2023 6:23 AM, legg wrote:
> On Tue, 28 Mar 2023 07:14:16 -0700, Don Y
> <blockedofcourse@foo.invalid> wrote:
>
>> On 3/28/2023 6:34 AM, legg wrote:
>>> On Mon, 27 Mar 2023 15:11:07 -0700, Don Y
>>> <blockedofcourse@foo.invalid> wrote:
>>>
>>>> On 3/26/2023 8:51 PM, Don Y wrote:
>>>>> I'm planning on operating a couple of 1U servers "on their sides"
>>>>> (i.e., tall, deep and VERY skinny).
>>>>>
>>>>> Of course, this is normal to their expected operating orientation.
>>>>>
>>>>> I figure the disks won't mind being operated in that orientation
>>>>> as I have shelfs with (the same) drives oriented that way.
>>>>>
>>>>> 1U servers tend to have a bank of redundant fans located behind
>>>>> the drive array to produce a flow through the drives and across
>>>>> the CPUs on its way out the vents in the rear. The fins of
>>>>> the CPU heatsinks are oriented to exploit this flow.
>>>>>
>>>>> But, the heatsinks also were designed with the expectation (?)
>>>>> that the fins would be upright (not sideways). Likewise, the
>>>>> CPU die expect the heatsinks to be "above". And, all the RAM
>>>>> to be in the same horizontal plane, etc.
>>>>>
>>>>> Of course, you can operate a CPU (or RAM, etc.) in any orientation.
>>>>> There are plenty of towers, etc. that are designed that way.
>>>>> And, machines that can be operated in either orientation.
>>>>>
>>>>> But, those are likely designed with that in mind!
>>>>>
>>>>> My question is: how tight is the thermal margin likely to
>>>>> be in these "already cramped" designs? Or, will my cold aisle
>>>>> likely more than compensate for any stresses?
>>>>>
>>>>> I suppose I could look at the temperatures that the two CPUs
>>>>> report and see if there is a noticeable difference...
>>>>
>>>> Verdict (friends who run data centers) is that I'm not going to
>>>> hammer on this hard enough to make a difference. Apparently,
>>>> they also make mounts for these in different orientations.
>>>>
>>>> So, we'll see...
>>>>
>>>
>>> You should follow bottom to top airflow, in a tall structure.
>>>
>>> If fans are unevenly locarted, use bottom to top as assist.
>>
>> That was what I would have assumed. But, the mounts that
>> are designed for vertical orientation (bottom-top, top-bottom...
>> as opposed to my "sideways" orientation) seem all to favor the front
>> of the server (cold aisle) at the top -- likely for ease of
>> human interaction (imagine bending down and craning to look
>> UP at the front panel; trying to insert an optical medium into
>> a tray that is opening *downward*, etc.)
>>
>> I assume that this is because the volume and rate of airflow through
>> the enclosure is >> what natural convection can counter.
>>
>> And, if that was the case, then (in my sideways orientation)
>> the same argument would apply to the CPU located "farther from
>> the floor" vs. the CPU *closest* to the floor. I.e., there may
>> be some temperature differences but nothing that would let
>> one CPU be in its SOA while the other was borderline.
>>
>> The fact that such servers are typically (by an overwhelming percentage)
>> deployed in the "flat" orientation led me to wonder if this
>> assumption was baked into the design...
>>
>> [I don't want to site a rack in the office]
>>
>
> Don't assume that the designer took this orientation into
> account, unless it's documented in the user/install manual.
The server makes no mention of orientation. In fact, I don't
see anything on any of the datasheets/manuals that I've
examined.
As for the "mounting enclosure" (bracket?)... the design
obviously intends cold aisle up. But, whether the needs
of the server were factored into that decision -- or simply
ergonomics -- is unknown.
> If you're fooling with it, just keep the rising air in mind,
> particularly if stacking.
>
> Convention is also aisle intake cool, rear exhaust hot.
Yes, but in a home, there's little choice in managing ambient.
(I'm not going to install supplemental cooling just to keep
the servers happy)
I will try to examine the CPU temperatures, while running,
to get a feel for whether or not one processor is seeing
a notably different temperature than the other (mounted
above/below it, vertically).
It's likely that I won't push the thermal envelope on the enclosure
as most of the disks will be idling (not seeking) at any given time
so the air feeding the CPUs shouldn't be as preheated as it would
likely be in a normal deployment.
OTOH, exhaust air won't be carried away but, rather, discharged
proximate to the intake.
<shrug> Servers are largely disposable; the folks who would likely
want them are businesses. And, businesses seldom want anything
other than "new". So, if I manage to toast these, I'll just
swap them out for two others -- after rethinking the mounting issue.
The 30 inch front-to-back dimension is the pisser. I'm mounting
these "on edge" as "bookends" under my benches (where I can
tolerate the extra "depth". The 24 inch devices are much easier
to accommodate in their normal orientation (32 inch deep benches)!
But, they tend to be less capable devices.
Reply by legg●March 29, 20232023-03-29
On Tue, 28 Mar 2023 07:14:16 -0700, Don Y
<blockedofcourse@foo.invalid> wrote:
>On 3/28/2023 6:34 AM, legg wrote:
>> On Mon, 27 Mar 2023 15:11:07 -0700, Don Y
>> <blockedofcourse@foo.invalid> wrote:
>>
>>> On 3/26/2023 8:51 PM, Don Y wrote:
>>>> I'm planning on operating a couple of 1U servers "on their sides"
>>>> (i.e., tall, deep and VERY skinny).
>>>>
>>>> Of course, this is normal to their expected operating orientation.
>>>>
>>>> I figure the disks won't mind being operated in that orientation
>>>> as I have shelfs with (the same) drives oriented that way.
>>>>
>>>> 1U servers tend to have a bank of redundant fans located behind
>>>> the drive array to produce a flow through the drives and across
>>>> the CPUs on its way out the vents in the rear.� The fins of
>>>> the CPU heatsinks are oriented to exploit this flow.
>>>>
>>>> But, the heatsinks also were designed with the expectation (?)
>>>> that the fins would be upright (not sideways).� Likewise, the
>>>> CPU die expect the heatsinks to be "above".� And, all the RAM
>>>> to be in the same horizontal plane, etc.
>>>>
>>>> Of course, you can operate a CPU (or RAM, etc.) in any orientation.
>>>> There are plenty of towers, etc. that are designed that way.
>>>> And, machines that can be operated in either orientation.
>>>>
>>>> But, those are likely designed with that in mind!
>>>>
>>>> My question is:� how tight is the thermal margin likely to
>>>> be in these "already cramped" designs?� Or, will my cold aisle
>>>> likely more than compensate for any stresses?
>>>>
>>>> I suppose I could look at the temperatures that the two CPUs
>>>> report and see if there is a noticeable difference...
>>>
>>> Verdict (friends who run data centers) is that I'm not going to
>>> hammer on this hard enough to make a difference. Apparently,
>>> they also make mounts for these in different orientations.
>>>
>>> So, we'll see...
>>>
>>
>> You should follow bottom to top airflow, in a tall structure.
>>
>> If fans are unevenly locarted, use bottom to top as assist.
>
>That was what I would have assumed. But, the mounts that
>are designed for vertical orientation (bottom-top, top-bottom...
>as opposed to my "sideways" orientation) seem all to favor the front
>of the server (cold aisle) at the top -- likely for ease of
>human interaction (imagine bending down and craning to look
>UP at the front panel; trying to insert an optical medium into
>a tray that is opening *downward*, etc.)
>
>I assume that this is because the volume and rate of airflow through
>the enclosure is >> what natural convection can counter.
>
>And, if that was the case, then (in my sideways orientation)
>the same argument would apply to the CPU located "farther from
>the floor" vs. the CPU *closest* to the floor. I.e., there may
>be some temperature differences but nothing that would let
>one CPU be in its SOA while the other was borderline.
>
>The fact that such servers are typically (by an overwhelming percentage)
>deployed in the "flat" orientation led me to wonder if this
>assumption was baked into the design...
>
>[I don't want to site a rack in the office]
>
Don't assume that the designer took this orientation into
account, unless it's documented in the user/install manual.
If you're fooling with it, just keep the rising air in mind,
particularly if stacking.
Convention is also aisle intake cool, rear exhaust hot.
RL
Reply by Don Y●March 28, 20232023-03-28
On 3/28/2023 4:18 PM, Lasse Langwadt Christensen wrote:
> onsdag den 29. marts 2023 kl. 01.12.01 UTC+2 skrev Don Y:
>> On 3/28/2023 1:52 PM, Clifford Heath wrote:
>>> On 29/03/23 04:32, Don Y wrote:
>>>> On 3/28/2023 8:29 AM, Phil Hobbs wrote:
>>>>> The natural-convection performance of the pizza box format is terrible,
>>>>> which (along with the small diameter fans) is why they sound like jet
>>>>> engines.
>>>>
>>>> Yes. So, you have to wonder why they haven't been able to quiet things down.
>>>> Does quieting the fans (only some of which can be done by mechanical
>>>> redesign of the fans) end up having too large an impact on airflow/cooling
>>>> capacity?
>>>
>>> Redesigning the fans for a 1U sized box doesn't help. They need to use larger
>>> diameter fans that spin slower, and those don't fit in 1U.
>> Which suggests that they *need* the fans that they have
>> and can't "throttle them down".
>>
>> I.e., active coolers can adjust their fan speeds. Yet, most servers
>> (esp low U numbers) tend to run the fans flat out.
>
> most I've used bootup sounding like a jet engine, but once they are booted they throttle
> the fans based on load, except when things like fancy graphic cards have been installed
> without the proper "blessings" from the system, then they sometimes run flat out
But they're never quiet. I have other 2U boxes that throttle the fans down
so you can't hear them -- until you start making the machines "work".
Likewise, the 40mm coolers on the CPUs in my workstations are never
audible -- despite being comparable processors to those in the servers.
Reply by Lasse Langwadt Christensen●March 28, 20232023-03-28
onsdag den 29. marts 2023 kl. 01.12.01 UTC+2 skrev Don Y:
> On 3/28/2023 1:52 PM, Clifford Heath wrote:
> > On 29/03/23 04:32, Don Y wrote:
> >> On 3/28/2023 8:29 AM, Phil Hobbs wrote:
> >>> The natural-convection performance of the pizza box format is terrible,
> >>> which (along with the small diameter fans) is why they sound like jet
> >>> engines.
> >>
> >> Yes. So, you have to wonder why they haven't been able to quiet things down.
> >> Does quieting the fans (only some of which can be done by mechanical
> >> redesign of the fans) end up having too large an impact on airflow/cooling
> >> capacity?
> >
> > Redesigning the fans for a 1U sized box doesn't help. They need to use larger
> > diameter fans that spin slower, and those don't fit in 1U.
> Which suggests that they *need* the fans that they have
> and can't "throttle them down".
>
> I.e., active coolers can adjust their fan speeds. Yet, most servers
> (esp low U numbers) tend to run the fans flat out.
most I've used bootup sounding like a jet engine, but once they are booted they throttle
the fans based on load, except when things like fancy graphic cards have been installed
without the proper "blessings" from the system, then they sometimes run flat out
Reply by Don Y●March 28, 20232023-03-28
On 3/28/2023 1:52 PM, Clifford Heath wrote:
> On 29/03/23 04:32, Don Y wrote:
>> On 3/28/2023 8:29 AM, Phil Hobbs wrote:
>>> The natural-convection performance of the pizza box format is terrible,
>>> which (along with the small diameter fans) is why they sound like jet
>>> engines.
>>
>> Yes. So, you have to wonder why they haven't been able to quiet things down.
>> Does quieting the fans (only some of which can be done by mechanical
>> redesign of the fans) end up having too large an impact on airflow/cooling
>> capacity?
>
> Redesigning the fans for a 1U sized box doesn't help. They need to use larger
> diameter fans that spin slower, and those don't fit in 1U.
Which suggests that they *need* the fans that they have
and can't "throttle them down".
I.e., active coolers can adjust their fan speeds. Yet, most servers
(esp low U numbers) tend to run the fans flat out.
Reply by Clifford Heath●March 28, 20232023-03-28
On 29/03/23 04:32, Don Y wrote:
> On 3/28/2023 8:29 AM, Phil Hobbs wrote:
>> The natural-convection performance of the pizza box format is terrible,
>> which (along with the small diameter fans) is why they sound like jet
>> engines.
>
> Yes. So, you have to wonder why they haven't been able to quiet things
> down.
> Does quieting the fans (only some of which can be done by mechanical
> redesign of the fans) end up having too large an impact on airflow/cooling
> capacity?
Redesigning the fans for a 1U sized box doesn't help. They need to use
larger diameter fans that spin slower, and those don't fit in 1U.
Reply by Don Y●March 28, 20232023-03-28
On 3/28/2023 8:29 AM, Phil Hobbs wrote:
> The natural-convection performance of the pizza box format is terrible,
> which (along with the small diameter fans) is why they sound like jet
> engines.
Yes. So, you have to wonder why they haven't been able to quiet things down.
Do they figure server rooms are, already, noisey so "what's the problem
with a little more?"
Does quieting the fans (only some of which can be done by mechanical
redesign of the fans) end up having too large an impact on airflow/cooling
capacity?
Or, are they just playing things safe (more air is better than less air)?
Then, you have to look at the margins.
On the one hand, budgets for data centers tend to be flusher than
home/personal machines. So, they could possibly afford to build in
more thermal margin (than, for example, a cheap "home PC")
OTOH, they might reason that data center staff will provide more
scheduled maintenance than would typically be seen in a home
product (how often do folks clean their filters, heat sinks, etc.)?
Finally, how much does the mechanical configuration and orientation
*actually* play into the thermal performance of the device.
<shrug> No $$ involved and I can always replace with other servers
so this will be a learning experience...
> Oriented vertically, you could at least get a bit of chimney effect along the
> flat sides.